mlsanityData to 7 Oct 2026

Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptions

arXiv 2408.08780 ↗First posted 16 Aug 2024cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

With the help of in-context learning (ICL), large language models (LLMs) have achieved impressive performance across various tasks. However, the function of descriptive instructions during ICL remains under-explored. In this work, we propose an ensemble prompt framework to describe the selection criteria of multiple in-context examples, and preliminary experiments on machine translation (MT) across six translation directions confirm that this framework boosts ICL performance. But to our surprise, LLMs might not care what the descriptions actually say, and the performance gain is primarily caused by the ensemble format, since it could lead to improvement even with random descriptive nouns. We further apply this new ensemble framework on a range of commonsense, math, logical reasoning and hallucination tasks with three LLMs and achieve promising results, suggesting again that designing a proper prompt format would be much more effective and efficient than paying effort into specific descriptions.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 16 Aug 2024.

Builds on

30 of 47 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  10. 2022In-context Learning and Induction Headscited by 930
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  12. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
18 more
  1. 2022A Survey on In-context Learningcited by 717
  2. 2022Transformers learn in-context by gradient descentcited by 714
  3. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  4. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  6. 2023Larger language models do in-context learning differentlycited by 386
  7. 2023In-Context Learning Creates Task Vectorscited by 302
  8. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  9. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 235
  10. 2022In-context Examples Selection for Machine Translationcited by 172
  11. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 161
  12. 2023Compositional Exemplars for In-context Learningcited by 143
  13. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  14. 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
  15. 2024Parallel Structures in Pre-training Data Yield In-Context Learningcited by 26
  16. 2023CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translationcited by 12
  17. 2024NICE: To Optimize In-Context Examples or Not?cited by 9
  18. 2024SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Pioneer Agent: Continual Improvement of Small Language Models in Productioncited by 1
  2. 2026The Decomposition Tax: LLM Pipelines Lose Up to 40 Accuracy Points at Their Own Interfacescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  2. 2024What do Large Language Models Need for Machine Translation Evaluation?
  3. 2025Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines
  4. 2023In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax
  5. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  6. 2024Refining Translations with LLMs: A Constraint-Aware Iterative Prompting Approach
  7. 2025The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
  8. 2024The Power of Adaptation: Boosting In-Context Learning through Adaptive Prompting