mlsanityData to 7 Oct 2026

APEER: Automatic Prompt Engineering Enhances Large Language Model Reranking

arXiv 2406.14449 ↗First posted 20 Jun 2024cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have significantly enhanced Information Retrieval (IR) across various modules, such as reranking. Despite impressive performance, current zero-shot relevance ranking with LLMs heavily relies on human prompt engineering. Existing automatic prompt engineering algorithms primarily focus on language modeling and classification tasks, leaving the domain of IR, particularly reranking, underexplored. Directly applying current prompt engineering algorithms to relevance ranking is challenging due to the integration of query and long passage pairs in the input, where the ranking complexity surpasses classification tasks. To reduce human effort and unlock the potential of prompt optimization in reranking, we introduce a novel automatic prompt engineering algorithm named APEER. APEER iteratively generates refined prompts through feedback and preference optimization. Extensive experiments with four LLMs and ten datasets demonstrate the substantial performance improvement of APEER over existing state-of-the-art (SoTA) manual prompts. Furthermore, we find that the prompts generated by APEER exhibit better transferability across diverse tasks and LLMs.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 53 of its 85 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

53 of 85 references.

References · 53

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
38 more
  1. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  2. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  3. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  4. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  5. 2019How Can We Know What Language Models Know?cited by 953
  6. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  8. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  9. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  10. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  11. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  12. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  13. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  14. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  15. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  16. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  17. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  18. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  19. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  20. 2020Overview of the TREC 2019 deep learning trackcited by 282
  21. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  22. 2019Multi-Stage Document Ranking with BERTcited by 200
  23. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  24. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  25. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  26. 2023RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!cited by 151
  27. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  28. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  29. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
  30. 2016Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbotscited by 103
  31. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  32. 2022A Study of the Attention Abnormality in Trojaned BERTscited by 47
  33. 2024Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitatecited by 46
  34. 2023Attention-Enhancing Backdoor Attacks Against BERT-based Modelscited by 32
  35. 2023AutoReP: Automatic ReLU Replacement for Fast Private Network Inferencecited by 29
  36. 2023Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspectivecited by 23
  37. 2023Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-rankercited by 20
  38. 2024Particle Filter SLAM for Vehicle Localizationcited by 20

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 6; 2026: 4.

28 papers.

Sort:

By area:

  1. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  2. 2025Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoningcited by 29
  3. 2024Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approachcited by 24
  4. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  5. 2024Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusioncited by 16
  6. 2024AdaPI: Facilitating DNN Model Adaptivity for Efficient Private Inference in Edge Computingcited by 15
  7. 2024Data Imputation using Large Language Model to Accelerate Recommendation Systemcited by 15
  8. 2024Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learningcited by 12
  9. 2024Cross-border Commodity Pricing Strategy Optimization via Mixed Neural Network for Time Series Analysiscited by 12
  10. 2024Deep Analysis of Time Series Data for Smart Grid Startup Strategies: A Transformer-LSTM-PSO Model Approachcited by 11
  11. 2024Advanced User Credit Risk Prediction Model using LightGBM, XGBoost and Tabnet with SMOTEENNcited by 10
  12. 2024Advanced AI Framework for Enhanced Detection and Assessment of Abdominal Trauma: Integrating 3D Segmentation with 2D CNN and RNN Modelscited by 9
16 more
  1. 2024ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillationcited by 9
  2. 2024Recording Brain Activity While Listening to Music Using Wearable EEG Devices Combined with Bidirectional Long Short-Term Memory Networkscited by 8
  3. 2024Integration of Mamba and Transformer -- MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamicscited by 7
  4. 2024In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminatescited by 7
  5. 2025A Survey of Automatic Prompt Optimization with Instruction-focused Heuristic-based Search Algorithmcited by 6
  6. 2024Exploring Prompt Engineering: A Systematic Review with SWOT Analysiscited by 5
  7. 2025A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Modelscited by 4
  8. 2025Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevancecited by 4
  9. 2025The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Modelscited by 3
  10. 2024Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Modelscited by 2
  11. 2024Enhanced Credit Score Prediction Using Ensemble Deep Learning Modelcited by 2
  12. 2026An Efficient Long-Context Ranking Architecture With Calibrated LLM Distillation: Application to Person-Job Fitcited by 1
  13. 2026Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signalscited by 1
  14. 2025Multimodal Foundation Model-Driven User Interest Modeling and Behavior Analysis on Short Video Platformscited by 0
  15. 2026Towards Spec Learning: Inference-Time Alignment from Preference Pairscited by 0
  16. 2026Mitigating Modality and Language-Style Gaps for Zero-Shot Video Moment Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking
  2. 2025LLM Optimization Unlocks Real-Time Pairwise Reranking
  3. 2025RankLLM: A Python Package for Reranking with LLMs
  4. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
  5. 2024Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers
  6. 2026Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning
  7. 2026Tool-Adaptive LLM Reranker
  8. 2026Querit-Reranker: Training Compact Multilingual Rerankers via Efficient Label-Free Distribution Adaptation ↗