fairseq: A Fast, Extensible Toolkit for Sequence Modeling

1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
4. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
5. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
6. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
7. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
8. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
9. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
10. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
11. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
12. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435

50 most cited of 1,486 · All 1,486 (JSON)

1. 2026MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression24 Sep 2026
2. 2026EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation25 Aug 2026
3. 2026Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokens28 Jul 2026
4. 2026Tokenizing Crosslingual Homographs20 Jul 2026
5. 2026T^2MLR: Transformer with Temporal Middle-Layer Recurrence16 Jul 2026
6. 2026Flexformer: Flexible Linear Transformer with Learnable Attention Kernel26 Jun 2026
7. 2026Online Dynamic Batching with Formal Guarantees for LLM Training18 Jun 2026
8. 2026SuTRA : Structurally-Unified Tokenization with Root Awareness5 Jun 2026
9. 2026The Word and the Way: Strategies for Domain-Specific BERT Pre-Training in German Medical NLP2 Jun 2026
10. 2026GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs29 May 2026
11. 2026Your Multimodal Speech Model Says I Have a Face for Radio28 May 2026
12. 2026DunbaaBERT: From Sacrifice to Semantics26 May 2026

50 newest of 1,486 · All 1,486 (JSON)

1. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
2. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 199
3. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
4. 2021Taming Sparsely Activated Transformer with Stochastic Expertscited by 112
5. 2022MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Servicescited by 37
6. 2023Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inferencecited by 36
7. 2021Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policycited by 32
8. 2024Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Expertscited by 31
9. 2021EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gatecited by 30
10. 2023Sparse Universal Transformercited by 28
11. 2024XMoE: Sparse Models with Fine-grained and Adaptive Expert Selectioncited by 27
12. 2023Sparse Backpropagation for MoE Trainingcited by 16

12 most cited of 22 · All 22 (JSON)

1. 2025Fine-grained MoE Load Balancing with Linear Programming21 Nov 2025
2. 2025X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms18 Aug 2025
3. 2024Efficient and Interpretable Grammatical Error Correction with Mixture of Experts30 Oct 2024
4. 2024A Survey on Mixture of Experts in Large Language Models26 Jun 2024
5. 2024Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts7 Apr 2024
6. 2024XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection27 Feb 2024
7. 2024Turn Waste into Worth: Rectifying Top-k Router of MoE17 Feb 2024
8. 2024MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts31 Jan 2024
9. 2023Sparse Universal Transformer11 Oct 2023
10. 2023Sparse Backpropagation for MoE Training1 Oct 2023
11. 2023Task-Based MoE for Multitask Multilingual Machine Translation30 Aug 2023
12. 2023Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference10 Mar 2023

12 newest of 22 · All 22 (JSON)

1. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
2. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
3. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 63
4. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
5. 2024Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspectivecited by 22
6. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Timecited by 20
7. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!cited by 10
8. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
9. 2024Discrete Diffusion Language Model for Efficient Text Summarizationcited by 7
10. 2023Duplex Diffusion Models Improve Speech-to-Speech Translationcited by 5
11. 2025Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processescited by 4
12. 2026Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?cited by 3

12 most cited of 15 · All 15 (JSON)

1. 2026Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?2 Apr 2026
2. 2026Sentence Curve Language Models2 Feb 2026
3. 2025Learnable Sampler Distillation for Discrete Diffusion Models24 Sep 2025
4. 2025Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes28 May 2025
5. 2024Discrete Modeling via Boundary Conditional Diffusion Processes29 Oct 2024
6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct 2024
7. 2024Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspective16 Oct 2024
8. 2024Discrete Diffusion Language Model for Efficient Text Summarization25 Jun 2024
9. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time14 Dec 2023
10. 2023DiffS2UT: A Semantic Preserving Diffusion Model for Textless Direct Speech-to-Speech Translation26 Oct 2023
11. 2023Duplex Diffusion Models Improve Speech-to-Speech Translation22 May 2023
12. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!8 May 2023

12 newest of 15 · All 15 (JSON)

1. 2023Fast Distributed Inference Serving for Large Language Modelscited by 143
2. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inferencecited by 124
3. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instancescited by 83
4. 2024D'ej`aVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Servingcited by 48
5. 2024D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Modelscited by 30
6. 2022HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognitioncited by 13
7. 2024BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inferencecited by 8
8. 2026KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMscited by 5
9. 2026GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMscited by 4
10. 2026GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Servingcited by 3
11. 2024ALISE: Accelerating Large Language Model Serving with Speculative Schedulingcited by 2
12. 2026Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compressioncited by 2

12 most cited of 15 · All 15 (JSON)

1. 2026MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression24 Sep 2026
2. 2026GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs29 May 2026
3. 2026Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression2 Apr 2026
4. 2026GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving26 Mar 2026
5. 2026KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs5 Feb 2026
6. 2025FLToP CTC: Frame-Level Token Pruning via Relative Threshold for Efficient and Memory-Saving Decoding on Diverse Platforms10 Oct 2025
7. 2024ALISE: Accelerating Large Language Model Serving with Speculative Scheduling31 Oct 2024
8. 2024BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference30 Oct 2024
9. 2024D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models18 Jun 2024
10. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference14 Mar 2024
11. 2024D'ej`aVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving4 Mar 2024
12. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instances27 Nov 2023

12 newest of 15 · All 15 (JSON)

1. 2023AudioChatLlama: Towards General-Purpose Speech Abilities for LLMscited by 59
2. 2022Reasoning Like Program Executorscited by 45
3. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examplescited by 29
4. 2022Multi-Step Deductive Reasoning Over Natural Language: An Empirical Study on Out-of-Distribution Generalisationcited by 15
5. 2022GPT-Neo for commonsense reasoning -- a theoretical and practical lenscited by 4
6. 2023VideoOFA: Two-Stage Pre-Training for Video-to-Text Generationcited by 3
7. 2023Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysiscited by 2
8. 2026NGDB-Zoo: Towards Efficient and Scalable Neural Graph Databases Trainingcited by 2
9. 2023Explanation Graph Generation via Generative Pre-training over Synthetic Graphscited by 1

1. 2026NGDB-Zoo: Towards Efficient and Scalable Neural Graph Databases Training25 Feb 2026
2. 2023AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs12 Nov 2023
3. 2023Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysis13 Jun 2023
4. 2023Explanation Graph Generation via Generative Pre-training over Synthetic Graphs1 Jun 2023
5. 2023VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation4 May 2023
6. 2022GPT-Neo for commonsense reasoning -- a theoretical and practical lens28 Nov 2022
7. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples22 Oct 2022
8. 2022Multi-Step Deductive Reasoning Over Natural Language: An Empirical Study on Out-of-Distribution Generalisation28 Jul 2022
9. 2022Reasoning Like Program Executors27 Jan 2022

1. 2021Hidden Backdoors in Human-Centric Language Modelscited by 83
2. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 81
3. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermarkcited by 78
4. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 69
5. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Modelscited by 33
6. 2021Defending Against Backdoor Attacks in Natural Language Generationcited by 30
7. 2021Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoningcited by 19
8. 2023BDMMT: Backdoor Sample Detection for Language Models through Model Mutation Testingcited by 13
9. 2024A Method to Facilitate Membership Inference Attacks in Deep Learning Modelscited by 6

1. 2024A Method to Facilitate Membership Inference Attacks in Deep Learning Models2 Jul 2024
2. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models28 May 2023
3. 2023Protecting Language Generation Models via Invisible Watermarking6 Feb 2023
4. 2023BDMMT: Backdoor Sample Detection for Language Models through Model Mutation Testing25 Jan 2023
5. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures9 Dec 2021
6. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec 2021
7. 2021Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning12 Jul 2021
8. 2021Defending Against Backdoor Attacks in Natural Language Generation3 Jun 2021
9. 2021Hidden Backdoors in Human-Centric Language Models1 May 2021

1. 2023Language Model Alignment with Elastic Resetcited by 34
2. 2025IPO: Your Language Model is Secretly a Preference Classifiercited by 17
3. 2021Non-Parametric Online Learning from Human Feedback for Neural Machine Translationcited by 8
4. 2025SimulPL: Aligning Human Preferences in Simultaneous Machine Translationcited by 7
5. 2025Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimationcited by 4
6. 2021Learning Feature Weights using Reward Modeling for Denoising Parallel Corporacited by 1

1. 2025Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimation27 Feb 2025
2. 2025IPO: Your Language Model is Secretly a Preference Classifier22 Feb 2025
3. 2025SimulPL: Aligning Human Preferences in Simultaneous Machine Translation2 Feb 2025
4. 2023Language Model Alignment with Elastic Reset6 Dec 2023
5. 2021Non-Parametric Online Learning from Human Feedback for Neural Machine Translation23 Sep 2021
6. 2021Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora11 Mar 2021

1. 2023Towards human-like spoken dialogue generation between AI agents from written dialoguecited by 19
2. 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkitcited by 9
3. 2024Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Modelscited by 2
4. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinationscited by 0

1. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations2 Jun 2025
2. 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkit21 Feb 2025
3. 2024Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models11 Jun 2024
4. 2023Towards human-like spoken dialogue generation between AI agents from written dialogue2 Oct 2023

1. 2023Image Captioners Are Scalable Vision Learners Toocited by 89
2. 2024Non-autoregressive Sequence-to-Sequence Vision-Language Modelscited by 5
3. 2025AudioCodecBench: A Comprehensive Benchmark for Audio Codec Evaluationcited by 0

1. 2025AudioCodecBench: A Comprehensive Benchmark for Audio Codec Evaluation2 Sep 2025
2. 2024Non-autoregressive Sequence-to-Sequence Vision-Language Models4 Mar 2024
3. 2023Image Captioners Are Scalable Vision Learners Too13 Jun 2023

1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
2. 2023Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Modelscited by 23
3. 2025Improving Retrieval-Augmented Neural Machine Translation with Monolingual Datacited by 0

1. 2025Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data30 Apr 2025
2. 2023Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models15 Oct 2023
3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May 2020

1. 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 130
2. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 68

1. 2021Proof Artifact Co-training for Theorem Proving with Language Models11 Feb 2021
2. 2020Autoregressive Knowledge Distillation through Imitation Learning15 Sep 2020

1. 2024Rethinking Mamba in Speech Processing by Self-Supervised Modelscited by 10
2. 2023State Spaces Aren't Enough: Machine Translation Needs Attentioncited by 9

1. 2024Rethinking Mamba in Speech Processing by Self-Supervised Models11 Sep 2024
2. 2023State Spaces Aren't Enough: Machine Translation Needs Attention25 Apr 2023

1. 2024Scrambled text: training Language Models to correct OCR errors using synthetic datacited by 2
2. 2025Learning Beyond Limits: Multitask Learning and Synthetic Data for Low-Resource Canonical Morpheme Segmentationcited by 0

1. 2025Learning Beyond Limits: Multitask Learning and Synthetic Data for Low-Resource Canonical Morpheme Segmentation22 May 2025
2. 2024Scrambled text: training Language Models to correct OCR errors using synthetic data29 Sep 2024

1. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modelingcited by 34

1. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modeling21 Feb 2025

1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
3. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
4. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
5. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
7. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
8. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
9. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
10. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
11. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
12. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404

12 most cited of 1,393 · All 1,393 (JSON)

1. 2026EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation25 Aug 2026
2. 2026Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokens28 Jul 2026
3. 2026Tokenizing Crosslingual Homographs20 Jul 2026
4. 2026T^2MLR: Transformer with Temporal Middle-Layer Recurrence16 Jul 2026
5. 2026Flexformer: Flexible Linear Transformer with Learnable Attention Kernel26 Jun 2026
6. 2026Online Dynamic Batching with Formal Guarantees for LLM Training18 Jun 2026
7. 2026SuTRA : Structurally-Unified Tokenization with Root Awareness5 Jun 2026
8. 2026The Word and the Way: Strategies for Domain-Specific BERT Pre-Training in German Medical NLP2 Jun 2026
9. 2026Your Multimodal Speech Model Says I Have a Face for Radio28 May 2026
10. 2026DunbaaBERT: From Sacrifice to Semantics26 May 2026
11. 2026Transformer Scalability Crisis: The First Comprehensive Empirical Analysis of Performance Walls in Modern Language Models14 May 2026
12. 2026Evolving Knowledge Distillation for Lightweight Neural Machine Translation11 May 2026

12 newest of 1,393 · All 1,393 (JSON)