{"total":87,"rows":[{"id":"2303.18223","title":"A Survey of Large Language Models","created":"2023-03-31","cited_by_count":2576,"area":null},{"id":"2211.05100","title":"BLOOM: A 176B-Parameter Open-Access Multilingual Language Model","created":"2022-11-09","cited_by_count":1984,"area":null},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103,"area":null},{"id":"2106.11959","title":"Revisiting Deep Learning Models for Tabular Data","created":"2021-06-22","cited_by_count":740,"area":null},{"id":"2310.01889","title":"Ring Attention with Blockwise Transformers for Near-Infinite Context","created":"2023-10-03","cited_by_count":457,"area":null},{"id":"2202.08906","title":"ST-MoE: Designing Stable and Transferable Sparse Expert Models","created":"2022-02-17","cited_by_count":399,"area":"moe"},{"id":"2105.03824","title":"FNet: Mixing Tokens with Fourier Transforms","created":"2021-05-09","cited_by_count":347,"area":null},{"id":"2106.05234","title":"Do Transformers Really Perform Bad for Graph Representation?","created":"2021-06-09","cited_by_count":320,"area":null},{"id":"2205.05131","title":"UL2: Unifying Language Learning Paradigms","created":"2022-05-10","cited_by_count":295,"area":null},{"id":"2202.08791","title":"cosFormer: Rethinking Softmax in Attention","created":"2022-02-17","cited_by_count":213,"area":"lr"},{"id":"2212.05055","title":"Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints","created":"2022-12-09","cited_by_count":209,"area":"moe"},{"id":"2202.10447","title":"Transformer Quality in Linear Time","created":"2022-02-21","cited_by_count":191,"area":null},{"id":"2302.12766","title":"Language-Driven Representation Learning for Robotics","created":"2023-02-24","cited_by_count":165,"area":null},{"id":"2109.08668","title":"Primer: Searching for Efficient Transformers for Language Modeling","created":"2021-09-17","cited_by_count":164,"area":null},{"id":"2304.09151","title":"UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining","created":"2023-04-18","cited_by_count":151,"area":null},{"id":"2204.05832","title":"What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?","created":"2022-04-12","cited_by_count":150,"area":null},{"id":"2209.01667","title":"A Review of Sparse Expert Models in Deep Learning","created":"2022-09-04","cited_by_count":148,"area":"moe"},{"id":"2106.02584","title":"Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning","created":"2021-06-04","cited_by_count":113,"area":null},{"id":"2109.06716","title":"HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO","created":"2021-09-14","cited_by_count":112,"area":null},{"id":"2109.10686","title":"Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers","created":"2021-09-22","cited_by_count":106,"area":null},{"id":"2207.10551","title":"Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?","created":"2022-07-21","cited_by_count":103,"area":null},{"id":"2110.12894","title":"The Efficiency Misnomer","created":"2021-10-25","cited_by_count":98,"area":null},{"id":"2310.04564","title":"ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models","created":"2023-10-06","cited_by_count":94,"area":null},{"id":"2111.12763","title":"Sparse is Enough in Scaling Transformers","created":"2021-11-24","cited_by_count":86,"area":null},{"id":"2403.07059","title":"Better than classical? The subtle art of benchmarking quantum machine learning models","created":"2024-03-11","cited_by_count":85,"area":null},{"id":"2111.08284","title":"Few-Shot Self-Rationalization with Natural Language Prompts","created":"2021-11-16","cited_by_count":83,"area":null},{"id":"2212.14034","title":"Cramming: Training a Language Model on a Single GPU in One Day","created":"2022-12-28","cited_by_count":81,"area":null},{"id":"2103.14636","title":"A Practical Survey on Faster and Lighter Transformers","created":"2021-03-26","cited_by_count":73,"area":null},{"id":"2106.10207","title":"Distributed Deep Learning in Open Collaborations","created":"2021-06-18","cited_by_count":58,"area":null},{"id":"2205.13401","title":"Your Transformer May Not be as Powerful as You Expect","created":"2022-05-26","cited_by_count":55,"area":null},{"id":"2106.12566","title":"Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding","created":"2021-06-23","cited_by_count":51,"area":null},{"id":"2212.10544","title":"Pretraining Without Attention","created":"2022-12-20","cited_by_count":51,"area":null},{"id":"2108.01682","title":"Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation","created":"2021-08-03","cited_by_count":41,"area":null},{"id":"2307.06440","title":"No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models","created":"2023-07-12","cited_by_count":41,"area":null},{"id":"2210.06640","title":"Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities","created":"2022-10-13","cited_by_count":38,"area":null},{"id":"2109.05644","title":"SHAPE: Shifted Absolute Position Embedding for Transformers","created":"2021-09-13","cited_by_count":37,"area":null},{"id":"2110.04725","title":"Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning","created":"2021-10-10","cited_by_count":35,"area":null},{"id":"2312.17482","title":"MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining","created":"2023-12-29","cited_by_count":34,"area":null},{"id":"2305.14858","title":"Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers","created":"2023-05-24","cited_by_count":30,"area":null},{"id":"2204.06815","title":"deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks","created":"2022-04-14","cited_by_count":28,"area":null},{"id":"2208.12852","title":"What Do NLP Researchers Believe? Results of the NLP Community Metasurvey","created":"2022-08-26","cited_by_count":28,"area":null},{"id":"2410.03730","title":"Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs","created":"2024-09-30","cited_by_count":22,"area":null},{"id":"2405.14428","title":"Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs","created":"2024-05-23","cited_by_count":21,"area":null},{"id":"2204.06644","title":"METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals","created":"2022-04-13","cited_by_count":20,"area":null},{"id":"2106.01151","title":"Towards Deeper Deep Reinforcement Learning with Spectral Normalization","created":"2021-06-02","cited_by_count":19,"area":null},{"id":"2203.03853","title":"Where Does the Performance Improvement Come From? -- A Reproducibility Concern about Image-Text Retrieval","created":"2022-03-08","cited_by_count":19,"area":null},{"id":"2405.02200","title":"Position: Why We Must Rethink Empirical Research in Machine Learning","created":"2024-05-03","cited_by_count":19,"area":null},{"id":"2303.16166","title":"When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP","created":"2023-03-28","cited_by_count":14,"area":null},{"id":"2205.12399","title":"Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT","created":"2022-05-24","cited_by_count":12,"area":"moe"},{"id":"2305.19370","title":"Blockwise Parallel Transformer for Large Context Models","created":"2023-05-30","cited_by_count":12,"area":null},{"id":"2206.05985","title":"Modeling the Machine Learning Multiverse","created":"2022-06-13","cited_by_count":11,"area":null},{"id":"2109.06679","title":"Efficient Inference for Multilingual Neural Machine Translation","created":"2021-09-14","cited_by_count":10,"area":null},{"id":"2309.11197","title":"The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute","created":"2023-09-20","cited_by_count":10,"area":null},{"id":"2405.08553","title":"Improving Transformers with Dynamically Composable Multi-Head Attention","created":"2024-05-14","cited_by_count":10,"area":null},{"id":"2305.01666","title":"BrainNPT: Pre-training of Transformer networks for brain network classification","created":"2023-05-02","cited_by_count":9,"area":null},{"id":"2211.16504","title":"Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles","created":"2022-11-29","cited_by_count":7,"area":"mllm"},{"id":"2406.11353","title":"$\\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts","created":"2024-06-17","cited_by_count":7,"area":"moe"},{"id":"2408.08073","title":"Extracting Sentence Embeddings from Pretrained Transformer Models","created":"2024-08-15","cited_by_count":7,"area":null},{"id":"2107.02095","title":"Are standard Object Segmentation models sufficient for Learning Affordance Segmentation?","created":"2021-07-05","cited_by_count":6,"area":null},{"id":"2403.00877","title":"Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation","created":"2024-03-01","cited_by_count":6,"area":null},{"id":"2311.04449","title":"Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability","created":"2023-11-08","cited_by_count":5,"area":null},{"id":"2201.07365","title":"Improving Neural Machine Translation by Denoising Training","created":"2022-01-19","cited_by_count":4,"area":null},{"id":"2210.14593","title":"Scaling Laws Beyond Backpropagation","created":"2022-10-26","cited_by_count":4,"area":null},{"id":"2210.15541","title":"Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost","created":"2022-10-27","cited_by_count":4,"area":null},{"id":"2406.01757","title":"Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities","created":"2024-06-03","cited_by_count":4,"area":null},{"id":"2409.10089","title":"Cross-modality image synthesis from TOF-MRA to CTA using diffusion-based models","created":"2024-09-16","cited_by_count":4,"area":"diffusion"},{"id":"2510.26622","title":"Encoder-Decoder or Decoder-Only? Revisiting Encoder-Decoder Large Language Model","created":"2025-10-30","cited_by_count":4,"area":null},{"id":"2504.16394","title":"ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphs","created":"2025-04-23","cited_by_count":3,"area":null},{"id":"2605.10504","title":"Learning Less Is More: Premature Upper-Layer Attention Specialization Hurts Language Model Pretraining","created":"2026-05-11","cited_by_count":2,"area":null},{"id":"2605.11405","title":"20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone","created":"2026-05-12","cited_by_count":2,"area":"mllm"},{"id":"2205.04376","title":"EigenNoise: A Contrastive Prior to Warm-Start Representations","created":"2022-05-09","cited_by_count":1,"area":null},{"id":"2303.16132","title":"Transformer and Snowball Graph Convolution Learning for Brain functional network Classification","created":"2023-03-28","cited_by_count":1,"area":null},{"id":"2508.20640","title":"CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Models","created":"2025-08-28","cited_by_count":1,"area":"diffusion"},{"id":"2205.00148","title":"To Know by the Company Words Keep and What Else Lies in the Vicinity","created":"2022-04-30","cited_by_count":0,"area":null},{"id":"2205.05842","title":"Supplementary Material: Implementation and Experiments for GAU-based Model","created":"2022-05-12","cited_by_count":0,"area":null},{"id":"2207.08766","title":"Word Play for Playing Othello (Reverses)","created":"2022-07-18","cited_by_count":0,"area":null},{"id":"2211.01486","title":"Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis","created":"2022-11-02","cited_by_count":0,"area":null},{"id":"2404.14395","title":"PARAMANU-GANITA: Can Small Math Language Models Rival with Large Language Models on Mathematical Reasoning?","created":"2024-04-22","cited_by_count":0,"area":"lr"},{"id":"2410.12292","title":"How much do contextualized representations encode long-range context?","created":"2024-10-16","cited_by_count":0,"area":null},{"id":"2503.24237","title":"Spatio-temporal Prediction of Fine-Grained Origin-Destination Matrices with Applications in Ridesharing","created":"2025-03-31","cited_by_count":0,"area":null},{"id":"2604.14430","title":"Three-Phase Transformer","created":"2026-04-15","cited_by_count":0,"area":null},{"id":"2604.23434","title":"When Does Removing LayerNorm Help? Activation Bounding as a Regime-Dependent Implicit Regularizer","created":"2026-04-25","cited_by_count":0,"area":null},{"id":"2605.24971","title":"TGFormer: Towards Temporal Graph Transformer with Auto-Correlation Mechanism","created":"2026-05-24","cited_by_count":0,"area":null},{"id":"2608.01283","title":"Riemannian Attention Mechanisms for Transformers: A Theoretical Framework and Architecture Design","created":"2026-08-02","cited_by_count":0,"area":null},{"id":"2608.08961","title":"Gradient Under Microscope: Benchmarking Resource Utilization of Memory-Efficient Gradient Computation Methods","created":"2026-08-09","cited_by_count":0,"area":null},{"id":"2609.22005","title":"Abstention and Noise Filtering: Two Missing Primitives of Softmax Attention","created":"2026-09-18","cited_by_count":0,"area":null},{"id":"2609.27201","title":"A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systems","created":"2026-09-23","cited_by_count":0,"area":null}],"next_cursor":null}