{"list":"other","total":80,"rows":[{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2304.08485","title":"Visual Instruction Tuning","created":"2023-04-17","cited_by_count":7869},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"2303.15343","title":"Sigmoid Loss for Language Image Pre-Training","created":"2023-03-27","cited_by_count":3109},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"2408.03326","title":"LLaVA-OneVision: Easy Visual Task Transfer","created":"2024-08-06","cited_by_count":2719},{"id":"1908.02265","title":"ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks","created":"2019-08-06","cited_by_count":2222},{"id":"2307.06281","title":"MMBench: Is Your Multi-modal Model an All-around Player?","created":"2023-07-12","cited_by_count":2141},{"id":"2311.16502","title":"MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI","created":"2023-11-27","cited_by_count":2080},{"id":"1808.06226","title":"SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing","created":"2018-08-19","cited_by_count":1989},{"id":"2209.09513","title":"Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering","created":"2022-09-20","cited_by_count":1919},{"id":"2401.04088","title":"Mixtral of Experts","created":"2024-01-08","cited_by_count":1823},{"id":"1811.00937","title":"CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge","created":"2018-11-02","cited_by_count":1822},{"id":"2305.10355","title":"Evaluating Object Hallucination in Large Vision-Language Models","created":"2023-05-17","cited_by_count":1729},{"id":"1904.08920","title":"Towards VQA Models That Can Read","created":"2019-04-18","cited_by_count":1718},{"id":"2306.13394","title":"MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models","created":"2023-06-23","cited_by_count":1652},{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631},{"id":"2310.02255","title":"MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts","created":"2023-10-03","cited_by_count":1614},{"id":"2203.10244","title":"ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning","created":"2022-03-19","cited_by_count":1479},{"id":"2405.04434","title":"DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model","created":"2024-05-07","cited_by_count":1135},{"id":"1606.06031","title":"The LAMBADA dataset: Word prediction requiring a broad discourse context","created":"2016-06-20","cited_by_count":992},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"2403.20330","title":"Are We on the Right Way for Evaluating Large Vision-Language Models?","created":"2024-03-29","cited_by_count":914},{"id":"2112.06905","title":"GLaM: Efficient Scaling of Language Models with Mixture-of-Experts","created":"2021-12-13","cited_by_count":897},{"id":"2401.06066","title":"DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models","created":"2024-01-11","cited_by_count":867},{"id":"1704.04683","title":"RACE: Large-scale ReAding Comprehension Dataset From Examinations","created":"2017-04-15","cited_by_count":861},{"id":"1603.07396","title":"A Diagram Is Worth A Dozen Images","created":"2016-03-24","cited_by_count":804},{"id":"2106.05974","title":"Scaling Vision with Sparse Mixture of Experts","created":"2021-06-10","cited_by_count":747},{"id":"2401.02385","title":"TinyLlama: An Open-Source Small Language Model","created":"2024-01-04","cited_by_count":618},{"id":"2310.14566","title":"HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models","created":"2023-10-23","cited_by_count":583},{"id":"1912.00582","title":"BLiMP: The Benchmark of Linguistic Minimal Pairs for English","created":"2019-12-02","cited_by_count":564},{"id":"2305.07895","title":"OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models","created":"2023-05-13","cited_by_count":476},{"id":"2201.05596","title":"DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale","created":"2022-01-14","cited_by_count":404},{"id":"2202.08906","title":"ST-MoE: Designing Stable and Transferable Sparse Expert Models","created":"2022-02-17","cited_by_count":399},{"id":"2407.12772","title":"LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models","created":"2024-07-17","cited_by_count":353},{"id":"2103.16716","title":"BASE Layers: Simplifying Training of Large, Sparse Models","created":"2021-03-30","cited_by_count":320},{"id":"2409.02060","title":"OLMoE: Open Mixture-of-Experts Language Models","created":"2024-09-03","cited_by_count":301},{"id":"2401.15947","title":"MoE-LLaVA: Mixture of Experts for Large Vision-Language Models","created":"2024-01-29","cited_by_count":300},{"id":"2408.13257","title":"MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?","created":"2024-08-23","cited_by_count":292},{"id":"1312.4314","title":"Learning Factored Representations in a Deep Mixture of Experts","created":"2013-12-16","cited_by_count":278},{"id":"2106.04426","title":"Hash Layers For Large Sparse Models","created":"2021-06-08","cited_by_count":237},{"id":"2211.15841","title":"MegaBlocks: Efficient Sparse Training with Mixture-of-Experts","created":"2022-11-29","cited_by_count":209},{"id":"2212.05055","title":"Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints","created":"2022-12-09","cited_by_count":209},{"id":"2402.11684","title":"ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models","created":"2024-02-18","cited_by_count":205},{"id":"2206.03382","title":"Tutel: Adaptive Mixture-of-Experts at Scale","created":"2022-06-07","cited_by_count":201},{"id":"2402.01739","title":"OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models","created":"2024-01-29","cited_by_count":187},{"id":"2410.05993","title":"Aria: An Open Multimodal Native Mixture-of-Experts Model","created":"2024-10-08","cited_by_count":138},{"id":"2308.04014","title":"Continual Pre-Training of Large Language Models: How to (re)warm your model?","created":"2023-08-08","cited_by_count":137},{"id":"2106.03760","title":"DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning","created":"2021-06-07","cited_by_count":115},{"id":"2103.13262","title":"FastMoE: A Fast Mixture-of-Expert Training System","created":"2021-03-24","cited_by_count":112},{"id":"2403.07816","title":"Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM","created":"2024-03-12","cited_by_count":109},{"id":"1511.02301","title":"The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations","created":"2015-11-07","cited_by_count":108},{"id":"2210.14793","title":"M$^3$ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design","created":"2022-10-26","cited_by_count":98},{"id":"2303.07226","title":"Scaling Vision-Language Models with Sparse Mixture of Experts","created":"2023-03-13","cited_by_count":88},{"id":"2401.16160","title":"LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs","created":"2024-01-29","cited_by_count":84},{"id":"2410.08245","title":"Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts","created":"2024-10-10","cited_by_count":70},{"id":"2407.21770","title":"MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts","created":"2024-07-31","cited_by_count":62},{"id":"2406.06563","title":"Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models","created":"2024-06-03","cited_by_count":59},{"id":"2405.05949","title":"CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts","created":"2024-05-09","cited_by_count":56},{"id":"2410.07348","title":"MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts","created":"2024-10-09","cited_by_count":56},{"id":"2410.07524","title":"Upcycling Large Language Models into Mixture of Experts","created":"2024-10-10","cited_by_count":49},{"id":"2405.03133","title":"Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training","created":"2024-05-06","cited_by_count":37},{"id":"2010.07344","title":"Temperature check: theory and practice for training models with softmax-cross-entropy losses","created":"2020-10-14","cited_by_count":31},{"id":"2310.10837","title":"Approximating Two-Layer Feedforward Networks for Efficient Transformers","created":"2023-10-16","cited_by_count":27},{"id":"2405.13997","title":"Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts","created":"2024-05-22","cited_by_count":24},{"id":"2402.02952","title":"On Least Square Estimation in Softmax Gating Mixture of Experts","created":"2024-02-05","cited_by_count":23},{"id":"2309.11568","title":"BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model","created":"2023-09-20","cited_by_count":19},{"id":"2405.03594","title":"Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment","created":"2024-05-06","cited_by_count":15},{"id":"2207.09094","title":"MoEC: Mixture of Expert Clusters","created":"2022-07-19","cited_by_count":14},{"id":"2410.16682","title":"Methods of improving LLM training stability","created":"2024-10-22","cited_by_count":13},{"id":"2408.03511","title":"MoExtend: Tuning New Experts for Modality and Task Extension","created":"2024-08-07","cited_by_count":12},{"id":"2406.19905","title":"Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model","created":"2024-06-28","cited_by_count":11},{"id":"2410.14574","title":"MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts","created":"2024-10-18","cited_by_count":9},{"id":"2406.11353","title":"$\\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts","created":"2024-06-17","cited_by_count":7},{"id":"2410.01610","title":"Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging","created":"2024-10-02","cited_by_count":6},{"id":"2403.17404","title":"Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study","created":"2024-03-26","cited_by_count":5},{"id":"2410.12258","title":"Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts","created":"2024-10-16","cited_by_count":5}],"next_cursor":null}