mlsanityData to 14 Oct 2026

Tree Prompting: Efficient Task Adaptation without Fine-Tuning

arXiv 2310.14034 ↗First posted 21 Oct 2023cs.CL cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Prompting language models (LMs) is the main interface for applying them to new tasks. However, for smaller LMs, prompting provides low accuracy compared to gradient-based finetuning. Tree Prompting is an approach to prompting which builds a decision tree of prompts, linking multiple LM calls together to solve a task. At inference time, each call to the LM is determined by efficiently routing the outcome of the previous call using the tree. Experiments on classification datasets show that Tree Prompting improves accuracy over competing methods and is competitive with fine-tuning. We also show that variants of Tree Prompting allow inspection of a model's decision-making process.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 26 of its 59 references that are in the map. First posted on arXiv 21 Oct 2023.

Builds on

26 of 59 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  6. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  7. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  8. 2019How Can We Know What Language Models Know?cited by 959
  9. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  10. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  11. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  12. 2023Large Language Model Guided Tree-of-Thoughtcited by 221
14 more
  1. 2021How Many Data Points is a Prompt Worth?cited by 198
  2. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  3. 2021Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Modelscited by 139
  4. 2019Optimal Sparse Decision Treescited by 105
  5. 2019Interpreting Adversarially Trained Convolutional Neural Networkscited by 75
  6. 2022Iteratively Prompt Pre-trained Language Models for Chain of Thoughtcited by 75
  7. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  8. 2022Describing Differences between Text Distributions with Natural Languagecited by 54
  9. 2023SummIt: Iterative Text Summarization via ChatGPTcited by 54
  10. 2023kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inferencecited by 47
  11. 2022Augmenting Interpretable Models with LLMs during Trainingcited by 38
  12. 2022PromptBoosting: Black-Box Text Classification with Ten Forward Passescited by 36
  13. 2023PREFER: Prompt Ensemble Learning via Feedback-Reflect-Refinecited by 28
  14. 2023Stealing the Decoding Algorithms of Language Modelscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 10; 2025: 3; 2026: 4.

18 papers.

Sort:

By area:

  1. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  2. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  3. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  4. 2025Text Generation Beyond Discrete Token Samplingcited by 21
  5. 2024Vector-ICL: In-context Learning with Continuous Vector Representationscited by 13
  6. 2024Crafting Interpretable Embeddings by Asking LLMs Questionscited by 11
  7. 2026Human-AI Co-design for Clinical Prediction Modelscited by 9
  8. 2024SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Modelscited by 8
  9. 2024Large Language Models are Interpretable Learnerscited by 7
  10. 2024Learning a Decision Tree Algorithm with Transformerscited by 6
  11. 2024Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summariescited by 4
  12. 2024Interpretable Next-token Prediction via the Generalized Induction Headcited by 3
6 more
  1. 2025Dataset Featurization: Uncovering Natural Language Features through Unsupervised Data Reconstructioncited by 3
  2. 2024PEDAL: Enhancing Greedy Decoding with Large Language Models using Diverse Exemplarscited by 1
  3. 2025Random Rule Forest (RRF): Interpretable and Manageable Ensembles of LLM-Generated Questions for Predicting Success from Unstructured Datacited by 1
  4. 2026Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generationcited by 0
  5. 2026Data-Efficient Adaptation of LLMs via Attention Head Reweightingcited by 0
  6. 2026SemTree: Judge-Aware Semantic Decision Trees for Inspectable and Repairable Rule Decisionscited by 0