# Tiny Titans: Can Smaller Large Language Models Punch Above Their Weight in the Real World for Meeting Summarization?

arXiv: https://arxiv.org/abs/2402.00841 · HTML: https://mlsanity.com/paper/2402.00841/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [Efficient Large Language Models: A Survey](https://mlsanity.com/paper/2312.03863/) (2023)
- [A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://mlsanity.com/paper/2305.18486/) (2023)
- [A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Tasks](https://mlsanity.com/paper/2310.04270/) (2023)
- [Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspective](https://mlsanity.com/paper/2310.19233/) (2023)

## Built on by (most cited first, 30 of 33)

- [A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations](https://mlsanity.com/paper/2407.04069/) (2024)
- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Models](https://mlsanity.com/paper/2411.00823/) (2024)
- [To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answering](https://mlsanity.com/paper/2403.01924/) (2024)
- [Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge](https://mlsanity.com/paper/2506.00777/) (2025)
- [Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance](https://mlsanity.com/paper/2502.00641/) (2025)
- [Query-OPT: Optimizing Inference of Large Language Models via Multi-Query Instructions in Meeting Summarization](https://mlsanity.com/paper/2403.00067/) (2024)
- [AdaptEval: Evaluating Large Language Models on Domain Adaptation for Text Summarization](https://mlsanity.com/paper/2407.11591/) (2024)
- [Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflows](https://mlsanity.com/paper/2505.24189/) (2025)
- [Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models](https://mlsanity.com/paper/2601.13260/) (2026)
- [Best Practices for Large Language Models in Radiology](https://mlsanity.com/paper/2412.01233/) (2024)
- [Summarizing Speech: A Comprehensive Survey](https://mlsanity.com/paper/2504.08024/) (2025)
- [AI Knowledge Assist: An Automated Approach for the Creation of Knowledge Bases for Conversational AI Agents](https://mlsanity.com/paper/2510.08149/) (2025)
- [Reproducibility Study of Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation](https://mlsanity.com/paper/2502.16242/) (2025)
- [Arctic-TILT. Business Document Understanding at Sub-Billion Scale](https://mlsanity.com/paper/2408.04632/) (2024)
- [Hyper-Compression: Model Compression via Hyperfunction](https://mlsanity.com/paper/2409.00592/) (2024)
- [Exploring the Readiness of Prominent Small Language Models for the Democratization of Financial Literacy](https://mlsanity.com/paper/2410.07118/) (2024)
- [Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue](https://mlsanity.com/paper/2508.05283/) (2025)
- [GreenServ: Energy-Efficient Context-Aware Dynamic Routing for Multi-Model LLM Inference](https://mlsanity.com/paper/2601.17551/) (2026)
- [Evaluating the Effectiveness of Cost-Efficient Large Language Models in Benchmark Biomedical Tasks](https://mlsanity.com/paper/2507.14045/) (2025)
- [DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization](https://mlsanity.com/paper/2510.05858/) (2025)
- [Multimodal RAG Enhanced Visual Description](https://mlsanity.com/paper/2508.09170/) (2025)
- [DACIP-RC: Domain Adaptive Continual Instruction Pre-Training via Reading Comprehension on Business Conversations](https://mlsanity.com/paper/2510.08152/) (2025)
- [A Large-Scale Multi-Dimensional Empirical Study of LLMs for Conversation Summarization](https://mlsanity.com/paper/2606.15974/) (2026)
- [When Better Turns Do Not Make Better Agents: Diagnosing the Gap Between Next-Turn Metrics and Workflow Success](https://mlsanity.com/paper/2609.21187/) (2026)
- [Hybrid Student-Teacher Large Language Model Refinement for Cancer Toxicity Symptom Extraction](https://mlsanity.com/paper/2408.04775/) (2024)
- [Structured List-Grounded Question Answering](https://mlsanity.com/paper/2410.03950/) (2024)
- [Grid2Guide: A* Enabled Small Language Model for Indoor Navigation](https://mlsanity.com/paper/2508.08100/) (2025)
- [Identifying and Addressing User-level Security Concerns in Smart Homes Using "Smaller" LLMs](https://mlsanity.com/paper/2509.19485/) (2025)
- [Dual-stage and Lightweight Patient Chart Summarization for Emergency Physicians](https://mlsanity.com/paper/2510.06263/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive capabilities to solve a wide range of tasks without being explicitly fine-tuned on task-specific datasets. However, deploying LLMs in the real world is not trivial, as it requires substantial computing resources. In this paper, we investigate whether smaller, compact LLMs are a good alternative to the comparatively Larger LLMs2 to address significant costs associated with utilizing LLMs in the real world. In this regard, we study the meeting summarization task in a real-world industrial environment and conduct extensive experiments by comparing the performance of fine-tuned compact LLMs (e.g., FLAN-T5, TinyLLaMA, LiteLLaMA) with zero-shot larger LLMs (e.g., LLaMA-2, GPT-3.5, PaLM-2). We observe that most smaller LLMs, even after fine-tuning, fail to outperform larger zero-shot LLMs in meeting summarization datasets. However, a notable exception is FLAN-T5 (780M parameters), which performs on par or even better than many zero-shot Larger LLMs (from 7B to above 70B parameters), while being significantly smaller. This makes compact LLMs like FLAN-T5 a suitable cost-efficient solution for real-world industrial deployment.