{"list":"other","total":80,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380},{"id":"1801.03924","title":"The Unreasonable Effectiveness of Deep Features as a Perceptual Metric","created":"2018-01-11","cited_by_count":11177},{"id":"2304.07193","title":"DINOv2: Learning Robust Visual Features without Supervision","created":"2023-04-14","cited_by_count":7974},{"id":"2111.06377","title":"Masked Autoencoders Are Scalable Vision Learners","created":"2021-11-11","cited_by_count":7403},{"id":"1503.03585","title":"Deep Unsupervised Learning using Nonequilibrium Thermodynamics","created":"2015-03-12","cited_by_count":7058},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2212.09748","title":"Scalable Diffusion Models with Transformers","created":"2022-12-19","cited_by_count":6054},{"id":"1907.05600","title":"Generative Modeling by Estimating Gradients of the Data Distribution","created":"2019-07-12","cited_by_count":4101},{"id":"1909.11942","title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations","created":"2019-09-26","cited_by_count":2790},{"id":"2202.05262","title":"Locating and Editing Factual Associations in GPT","created":"2022-02-10","cited_by_count":2691},{"id":"2106.08254","title":"BEiT: BERT Pre-Training of Image Transformers","created":"2021-06-15","cited_by_count":2375},{"id":"2212.06817","title":"RT-1: Robotics Transformer for Real-World Control at Scale","created":"2022-12-13","cited_by_count":2061},{"id":"2106.01345","title":"Decision Transformer: Reinforcement Learning via Sequence Modeling","created":"2021-06-02","cited_by_count":1546},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"2309.08600","title":"Sparse Autoencoders Find Highly Interpretable Features in Language Models","created":"2023-09-15","cited_by_count":1420},{"id":"2005.10242","title":"Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere","created":"2020-05-20","cited_by_count":1405},{"id":"2012.14913","title":"Transformer Feed-Forward Layers Are Key-Value Memories","created":"2020-12-29","cited_by_count":1336},{"id":"2211.00593","title":"Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small","created":"2022-11-01","cited_by_count":1165},{"id":"2202.04200","title":"MaskGIT: Masked Generative Image Transformer","created":"2022-02-08","cited_by_count":1099},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"1503.02406","title":"Deep Learning and the Information Bottleneck Principle","created":"2015-03-09","cited_by_count":802},{"id":"2103.17239","title":"Going deeper with Image Transformers","created":"2021-03-31","cited_by_count":764},{"id":"1906.01563","title":"Hamiltonian Neural Networks","created":"2019-06-04","cited_by_count":712},{"id":"2406.11838","title":"Autoregressive Image Generation without Vector Quantization","created":"2024-06-17","cited_by_count":651},{"id":"2404.19756","title":"KAN: Kolmogorov-Arnold Networks","created":"2024-04-30","cited_by_count":649},{"id":"2301.00704","title":"Muse: Text-To-Image Generation via Masked Generative Transformers","created":"2023-01-02","cited_by_count":648},{"id":"1909.01377","title":"Deep Equilibrium Models","created":"2019-09-03","cited_by_count":615},{"id":"1703.00443","title":"OptNet: Differentiable Optimization as a Layer in Neural Networks","created":"2017-03-01","cited_by_count":599},{"id":"2008.02217","title":"Hopfield Networks is All You Need","created":"2020-07-16","cited_by_count":475},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"2502.05171","title":"Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach","created":"2025-02-07","cited_by_count":415},{"id":"2403.03853","title":"ShortGPT: Layers in Large Language Models are More Redundant Than You Expect","created":"2024-03-06","cited_by_count":383},{"id":"2010.04245","title":"Query-Key Normalization for Transformers","created":"2020-10-08","cited_by_count":280},{"id":"2003.04887","title":"ReZero is All You Need: Fast Convergence at Large Depth","created":"2020-03-10","cited_by_count":264},{"id":"1606.01164","title":"Dense Associative Memory for Pattern Recognition","created":"2016-06-03","cited_by_count":236},{"id":"2211.09117","title":"MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis","created":"2022-11-16","cited_by_count":227},{"id":"2403.17887","title":"The Unreasonable Ineffectiveness of the Deeper Layers","created":"2024-03-26","cited_by_count":226},{"id":"2204.10532","title":"End-to-end symbolic regression with transformers","created":"2022-04-22","cited_by_count":203},{"id":"1810.13400","title":"Differentiable MPC for End-to-end Planning and Control","created":"2018-10-31","cited_by_count":189},{"id":"1905.12149","title":"SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver","created":"2019-05-29","cited_by_count":173},{"id":"2107.05407","title":"PonderNet: Learning to Ponder","created":"2021-07-12","cited_by_count":168},{"id":"2406.19384","title":"The Remarkable Robustness of LLMs: Stages of Inference?","created":"2024-06-27","cited_by_count":146},{"id":"2012.08405","title":"Model-Based Deep Learning","created":"2020-12-15","cited_by_count":141},{"id":"2305.05465","title":"The emergence of clusters in self-attention dynamics","created":"2023-05-09","cited_by_count":140},{"id":"2110.11773","title":"Sinkformers: Transformers with Doubly Stochastic Attention","created":"2021-10-22","cited_by_count":138},{"id":"2209.02535","title":"Analyzing Transformers in Embedding Space","created":"2022-09-06","cited_by_count":132},{"id":"2106.04537","title":"Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks","created":"2021-06-08","cited_by_count":117},{"id":"2008.06996","title":"Large Associative Memory Problem in Neurobiology and Machine Learning","created":"2020-08-16","cited_by_count":112},{"id":"2306.01129","title":"White-Box Transformers via Sparse Rate Reduction","created":"2023-06-01","cited_by_count":80},{"id":"2410.20672","title":"Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA","created":"2024-10-28","cited_by_count":78},{"id":"1805.09298","title":"Learning towards Minimum Hyperspherical Energy","created":"2018-05-23","cited_by_count":75},{"id":"1711.08028","title":"Recurrent Relational Networks","created":"2017-11-21","cited_by_count":72},{"id":"2302.07253","title":"Energy Transformer","created":"2023-02-14","cited_by_count":71},{"id":"2410.01131","title":"nGPT: Normalized Transformer with Representation Learning on the Hypersphere","created":"2024-10-01","cited_by_count":70},{"id":"1808.10805","title":"Spherical Latent Spaces for Stable Variational Autoencoders","created":"2018-08-31","cited_by_count":67},{"id":"2103.06064","title":"Graph Neural Networks Inspired by Classical Iterative Algorithms","created":"2021-03-10","cited_by_count":60},{"id":"2405.16039","title":"MoEUT: Mixture-of-Experts Universal Transformers","created":"2024-05-25","cited_by_count":59},{"id":"2306.02572","title":"Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence","created":"2023-06-05","cited_by_count":56},{"id":"2407.09298","title":"Transformer Layers as Painters","created":"2024-07-12","cited_by_count":54},{"id":"1607.08194","title":"Convolutional Neural Networks Analyzed via Convolutional Sparse Coding","created":"2016-07-27","cited_by_count":52},{"id":"2305.16102","title":"Demystifying Oversmoothing in Attention-Based Graph Neural Networks","created":"2023-05-25","cited_by_count":49},{"id":"2402.04520","title":"On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis","created":"2024-02-07","cited_by_count":41},{"id":"2303.06562","title":"ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond","created":"2023-03-12","cited_by_count":39},{"id":"2404.03827","title":"Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models","created":"2024-04-04","cited_by_count":38},{"id":"2410.13821","title":"Artificial Kuramoto Oscillatory Neurons","created":"2024-10-17","cited_by_count":38},{"id":"2406.11179","title":"Learning Iterative Reasoning through Energy Diffusion","created":"2024-06-17","cited_by_count":36},{"id":"2203.06345","title":"The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy","created":"2022-03-12","cited_by_count":35},{"id":"2206.15448","title":"Learning Iterative Reasoning through Energy Minimization","created":"2022-06-30","cited_by_count":32},{"id":"2310.07096","title":"Sparse Universal Transformer","created":"2023-10-11","cited_by_count":32},{"id":"2307.04895","title":"Learning to Solve Constraint Satisfaction Problems with Recurrent Transformer","created":"2023-07-10","cited_by_count":21},{"id":"2410.23126","title":"Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes","created":"2024-10-30","cited_by_count":19},{"id":"2411.17182","title":"An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models","created":"2024-11-26","cited_by_count":4},{"id":"2412.00359","title":"Does Self-Attention Need Separate Weights in Transformers?","created":"2024-11-30","cited_by_count":4}],"next_cursor":null}