mlsanityData to 7 Oct 2026

LLMBox: A Comprehensive Library for Large Language Models

arXiv 2407.05563 ↗First posted 8 Jul 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

To facilitate the research on large language models (LLMs), this paper presents a comprehensive and unified library, LLMBox, to ease the development, use, and evaluation of LLMs. This library is featured with three main merits: (1) a unified data interface that supports the flexible implementation of various training strategies, (2) a comprehensive evaluation that covers extensive tasks, datasets, and models, and (3) more practical consideration, especially on user-friendliness and efficiency. With our library, users can easily reproduce existing methods, train new models, and conduct comprehensive performance comparisons. To rigorously test LLMBox, we conduct extensive experiments in a diverse coverage of evaluation settings, and experimental results demonstrate the effectiveness and efficiency of our library in supporting various implementations related to LLMs. The detailed introduction and usage guidance can be found at https://github.com/RUCAIBox/LLMBox.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 60 of its 79 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

60 of 79 references.

References · 60

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  11. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
38 more
  1. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  2. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  12. 2023A Survey of Large Language Modelscited by 2,537
  13. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  14. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  15. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  16. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  17. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  18. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  19. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  20. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  21. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  22. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  23. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  24. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  25. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  26. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  27. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  28. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  29. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  30. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  31. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  32. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  33. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  34. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  35. 2024Yi: Open Foundation Models by 01.AIcited by 706
  36. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  37. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  38. 2023Baichuan 2: Open Large-scale Language Modelscited by 664

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Modelscited by 5
  2. 2026A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Designcited by 3
  3. 2026Unified Deployment-Aware Evaluation of Open Reasoning Language Modelscited by 1