mlsanityData to 7 Oct 2026

Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together

arXiv 2407.10930 ↗First posted 15 Jul 2024cs.CL cs.AI cs.LGcited by 38

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Natural Language Processing (NLP) systems are increasingly taking the form of sophisticated modular pipelines, e.g., Retrieval Augmented Generation (RAG), where each module may involve a distinct Language Model (LM) and an associated prompt template. These compound systems often lack intermediate labels or gradient flow to optimize each module, making their end-to-end optimization challenging. Here we seek strategies to optimize both the module-level LM weights and the associated prompt templates of such systems to maximize a downstream task metric. We propose for the first time combining the weight and prompt optimization strategies to optimize a modular LM pipeline by alternating between the two to get the same LM to teach itself. In experiments with multi-hop QA, mathematical reasoning, and feature-based classification using mistral-7b, llama-2-7b, and llama-3-8b, these BetterTogether strategies optimizing the weights and prompts of a pipeline together outperform directly optimizing weights alone and prompts alone by up to 60% and 6%, respectively, on average across LMs and tasks. BetterTogether optimizer is released in DSPy at http://dspy.ai

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 15 of its 26 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

15 of 26 references.

References · 15

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  8. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  9. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  10. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  12. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
3 more
  1. 2021Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrievalcited by 63
  2. 2020Answering Open-Domain Questions of Varying Reasoning Steps from Textcited by 43
  3. 2023Prompt2Model: Generating Deployable Models from Natural Language Instructionscited by 40

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 25; 2026: 11.

38 papers.

Sort:

By area:

  1. 2024StructuredRAG: JSON Response Formatting with Large Language Modelscited by 22
  2. 2024Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrievalcited by 17
  3. 2025Tuning LLM Judge Design Decisions for 1/1000 of the Costcited by 11
  4. 2025Abacus: A Cost-Based Optimizer for Semantic Operator Systemscited by 11
  5. 2025PRL: Prompts from Reinforcement Learningcited by 9
  6. 2025DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discoverycited by 9
  7. 2026WHALE: A Simple Recipe for Joint Harness-Weight Optimizationcited by 9
  8. 2025A Survey of Automatic Prompt Optimization with Instruction-focused Heuristic-based Search Algorithmcited by 6
  9. 2025What Makes a Good Natural Language Prompt?cited by 6
  10. 2025promptolution: A Unified, Modular Framework for Prompt Optimizationcited by 6
  11. 2026Learning, Fast and Slow: Towards LLMs That Adapt Continuallycited by 6
  12. 2025Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directionscited by 4
26 more
  1. 2025Composing Policy Gradients and Prompt Optimization for Language Model Programscited by 4
  2. 2025Code-enabled language models can outperform reasoning models on diverse taskscited by 4
  3. 2025Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogscited by 4
  4. 2025Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challengecited by 3
  5. 2025MedVAL: Toward Expert-Level Medical Text Validation with Language Modelscited by 3
  6. 2025TextBO: Bayesian Optimization in Language Space for Eval-Efficient Self-Improving AIcited by 3
  7. 2025Clinical trial cohort selection using Large Language Models on n2c2 Challengescited by 2
  8. 2025Prompt and Parameter Co-Optimization for Large Language Modelscited by 2
  9. 2025Agentic Policy Optimization via Instruction-Policy Co-Evolutioncited by 2
  10. 2025PIAST: Rapid Prompting with In-context Augmentation for Scarce Training datacited by 2
  11. 2026OpenJarvis: Personal AI, On Personal Devicescited by 2
  12. 2025LangProBe: a Language Programs Benchmarkcited by 1
  13. 2025COSMOS: Predictable and Cost-Effective Adaptation of LLMscited by 1
  14. 2025Accelerating Automatic Program Repair with Dual Retrieval-Augmented Fine-Tuning and Patch Generation on Large Language Modelscited by 1
  15. 2025Type-Compliant Adaptation Cascades: Adapting Programmatic LM Workflows to Datacited by 1
  16. 2026Neurosymbolic LoRA: Why and When to Tune Weights vs. Rewrite Promptscited by 1
  17. 2026Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoningcited by 1
  18. 2026TATRA: Training-Free Instance-Adaptive Prompting Through Rephrasing and Aggregationcited by 1
  19. 2025Data Diversification Methods In Alignment Enhance Math Performance In LLMscited by 0
  20. 2025A Large-Scale Dataset and Citation Intent Classification in Turkish with LLMscited by 0
  21. 2025WRAVAL -- WRiting Assist eVALuationcited by 0
  22. 2026Learning Evidence Highlighting for Frozen LLMscited by 0
  23. 2026Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learningcited by 0
  24. 2026Prompt-Driven Exploration: Language as an Exploration Space for VLA Reinforcement Learningcited by 0
  25. 2026Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Promptscited by 0
  26. 2026ClusterFewshot: Improving Few-shot Optimization for LLMs workflowcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Supervised Prompt Optimization
  2. 2024GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers
  3. 2025TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards
  4. 2026Prompt2Skill: Unsupervised Skill Optimization From Natural Language Instructions
  5. 2025SIPDO: Closed-Loop Prompt Optimization via Synthetic Data Feedback
  6. 2024Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach
  7. 2026Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach
  8. 2025Optimising Language Models for Downstream Tasks: A Post-Training Perspective