mlsanityData to 7 Oct 2026

Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications

arXiv 2310.14103 ↗First posted 21 Oct 2023cs.LG cs.AI cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Instruction Fine-Tuning (IFT) is a powerful paradigm that strengthens the zero-shot capabilities of Large Language Models (LLMs), but in doing so induces new evaluation metric requirements. We show LLM-based metrics to be well adapted to these requirements, and leverage them to conduct an investigation of task-specialization strategies, quantifying the trade-offs that emerge in practical industrial settings. Our findings offer practitioners actionable insights for real-world IFT model deployment.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 21 Oct 2023.

Builds on

38 of 54 references.

References · 38

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
26 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  2. 2020Learning to summarize from human feedbackcited by 2,562
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  8. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  9. 2023LIMA: Less Is More for Alignmentcited by 1,080
  10. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  11. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  13. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  14. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  15. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  16. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  17. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  18. 2023The False Promise of Imitating Proprietary LLMscited by 254
  19. 2021How Many Data Points is a Prompt Worth?cited by 197
  20. 2020Re-evaluating Evaluation in Text Summarizationcited by 144
  21. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 73
  22. 2021Automatic Text Evaluation through the Lens of Wasserstein Barycenterscited by 38
  23. 2021InfoLM: A New Metric to Evaluate Summarization & Data2Text Generationcited by 33
  24. 2021A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regionscited by 17
  25. 2023Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarkscited by 14
  26. 2022Reproducibility Issues for BERT-based Evaluation Metricscited by 9

Built on by

Cited by year

Citing papers per year: 2024: 8; 2025: 3.

11 papers.

  1. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  2. 2024Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMscited by 61
  3. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 52
  4. 2024SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domaincited by 39
  5. 2025Just a Scratch: Enhancing LLM Capabilities for Self-harm Detection through Intent Differentiation and Emoji Interpretationcited by 7
  6. 2024GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answeringcited by 5
  7. 2024Mitigating the Impact of Reference Quality on Evaluation of Summarization Systems with Reference-Free Metricscited by 5
  8. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Modelscited by 5
  9. 2024Optimizing RAG Techniques for Automotive Industry PDF Chatbots: A Case Study with Locally Deployed Ollama Modelscited by 4
  10. 2024Synthesizing Conversations from Unlabeled Documents using Automatic Response Segmentationcited by 2
  11. 2025What is Beneath Misogyny: Misogynous Memes Classification and Explanationcited by 0