mlsanityData to 14 Oct 2026

Joint Prompt Optimization of Stacked LLMs using Variational Inference

arXiv 2306.12509 ↗First posted 21 Jun 2023cs.CL cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) can be seen as atomic units of computation mapping sequences to a distribution over sequences. Thus, they can be seen as stochastic language layers in a language network, where the learnable parameters are the natural language prompts at each layer. By stacking two such layers and feeding the output of one layer to the next, we obtain a Deep Language Network (DLN). We first show how to effectively perform prompt optimization for a 1-Layer language network (DLN-1). Then, we present an extension that applies to 2-layer DLNs (DLN-2), where two prompts must be learned. The key idea is to consider the output of the first layer as a latent variable, which requires inference, and prompts to be learned as the parameters of the generative distribution. We first test the effectiveness of DLN-1 in multiple reasoning and natural language understanding tasks. Then, we show that DLN-2 can reach higher performance than a single layer, showing promise that we might reach comparable performance to GPT-4, even when each LLM in the network is smaller and less powerful.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 49 of its 56 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

49 of 56 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  9. 2013Auto-Encoding Variational Bayescited by 4,053
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
37 more
  1. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  4. 2016Variational Inference: A Review for Statisticianscited by 1,472
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  6. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  8. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  9. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  10. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  11. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  13. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  14. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  15. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  16. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  17. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  18. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  19. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  20. 2023Augmented Language Models: a Surveycited by 403
  21. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  22. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 334
  23. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
  24. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 243
  25. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 214
  26. 2019Distilling Task-Specific Knowledge from BERT into Simple Neural Networkscited by 153
  27. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  28. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  29. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  30. 2022Language Model Cascadescited by 107
  31. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 84
  32. 2021Understanding by Understanding Not: Modeling Negation in Language Modelscited by 73
  33. 2022Recitation-Augmented Language Modelscited by 65
  34. 2019Learning to Few-Shot Learn Across Diverse Natural Language Classification Taskscited by 57
  35. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 47
  36. 2020XtremeDistil: Multi-stage Distillation for Massive Multilingual Modelscited by 38
  37. 2016Language as a Latent Variable: Discrete Generative Models for Sentence Compressioncited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 11; 2025: 11; 2026: 5.

32 papers.

Sort:

By area:

  1. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  2. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  3. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  4. 2024Symbolic Learning Enables Self-Evolving Agentscited by 106
  5. 2023Amortizing intractable inference in large language modelscited by 97
  6. 2026Privileged Information Distillation for Language Modelscited by 90
  7. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
  8. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 69
  9. 2024PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensemblescited by 57
  10. 2024Privacy-Preserving Retrieval-Augmented Generation with Differential Privacycited by 39
  11. 2025Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Datacited by 29
  12. 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
20 more
  1. 2024CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generationcited by 18
  2. 2025On the Effect of Sampling Diversity in Scaling LLM Inferencecited by 18
  3. 2024Verbalized Machine Learning: Revisiting Machine Learning with Language Modelscited by 17
  4. 2025Beyond Outlining: Heterogeneous Recursive Planning for Adaptive Long-form Writing with Language Modelscited by 16
  5. 2024Paraphrase and Solve: Exploring and Exploiting the Impact of Surface Form on Mathematical Reasoning in Large Language Modelscited by 14
  6. 2024No Free Lunch Theorem for Privacy-Preserving LLM Inferencecited by 13
  7. 2023Ask more, know better: Reinforce-Learned Prompt Questions for Decision Making with Large Language Modelscited by 9
  8. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Modelscited by 9
  9. 2025When AI Meets the Web: Prompt Injection Risks in Third-Party AI Chatbot Pluginscited by 9
  10. 2025A Survey of Automatic Prompt Optimization with Instruction-focused Heuristic-based Search Algorithmcited by 6
  11. 2025Scaling Textual Gradients via Sampling-Based Momentumcited by 6
  12. 2026Learning, Fast and Slow: Towards LLMs That Adapt Continuallycited by 6
  13. 2025Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directionscited by 4
  14. 2024Test-Time Fairness and Robustness in Large Language Modelscited by 3
  15. 2025DAST: Difficulty-Aware Self-Training on Large Language Modelscited by 3
  16. 2025One Rank at a Time: Cascading Error Dynamics in Sequential Learningcited by 3
  17. 2026DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generationcited by 1
  18. 2023Inductive-bias Learning: Generating Code Models with Large Language Modelcited by 0
  19. 2026Structured Nonparametric Variational Inference for Dependent Latent Modelingcited by 0
  20. 2026Towards Spec Learning: Inference-Time Alignment from Preference Pairscited by 0