# Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization

arXiv: https://arxiv.org/abs/2410.18624 · HTML: https://mlsanity.com/paper/2410.18624/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM](https://mlsanity.com/paper/2104.04473/) (2021)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](https://mlsanity.com/paper/2311.16079/) (2023)
- [DialogSum: A Real-Life Scenario Dialogue Summarization Dataset](https://mlsanity.com/paper/2105.06762/) (2021)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [FineSurE: Fine-grained Summarization Evaluation using LLMs](https://mlsanity.com/paper/2407.00908/) (2024)
- [ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change](https://mlsanity.com/paper/2401.09646/) (2024)
- [Following Length Constraints in Instructions](https://mlsanity.com/paper/2406.17744/) (2024)
- [Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trends](https://mlsanity.com/paper/2406.03487/) (2024)
- [Prompt-Based Length Controlled Generation with Multiple Control Types](https://mlsanity.com/paper/2406.10278/) (2024)
- [An End-to-End Dialogue Summarization System for Sales Calls](https://mlsanity.com/paper/2204.12951/) (2022)
- [On The Persona-based Summarization of Domain-Specific Documents](https://mlsanity.com/paper/2406.03986/) (2024)

## Built on by (most cited first, 3 of 3)

- [Summarizing Speech: A Comprehensive Survey](https://mlsanity.com/paper/2504.08024/) (2025)
- [You need to MIMIC to get FAME: Solving Meeting Transcript Scarcity with a Multi-Agent Conversations](https://mlsanity.com/paper/2502.13001/) (2025)
- [Spot the BlindSpots: Systematic Identification and Quantification of Fine-Grained LLM Biases in Contact Center Summaries](https://mlsanity.com/paper/2508.13124/) (2025)

## Abstract (arXiv)

This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs). Our approach involves initial experiments with prompting existing LLMs to generate summaries of telephone conversations, followed by the creation of a tailored synthetic training dataset utilizing stronger frontier models. We place special focus on the diversity of the generated data and on the ability to control the length of the generated summaries to meet various use-case specific requirements. The effectiveness of our method is evaluated using two state-of-the-art LLM-as-a-judge-based evaluation techniques to ensure the quality and relevance of the summaries. Our results show that fine-tuned Llama-2-7B-based summarization model performs on-par with GPT-4 in terms of factual accuracy, completeness and conciseness. Our findings demonstrate the potential for quickly bootstrapping a practical and efficient call summarization system.