mlsanityData to 14 Oct 2026

Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models

arXiv 2312.02969 ↗First posted 5 Dec 2023cs.CL cs.IRcited by 14

Not on the map. Why?

Abstract (arXiv)

Listwise rerankers based on large language models (LLM) are the zero-shot state-of-the-art. However, current works in this direction all depend on the GPT models, making it a single point of failure in scientific reproducibility. Moreover, it raises the concern that the current research findings only hold for GPT models but not LLM in general. In this work, we lift this pre-condition and build for the first time effective listwise rerankers without any form of dependency on GPT. Our passage retrieval experiments show that our best list se reranker surpasses the listwise rerankers based on GPT-3.5 by 13% and achieves 97% effectiveness of the ones built on GPT-4. Our results also show that the existing training datasets, which were expressly constructed for pointwise ranking, are insufficient for building such listwise rerankers. Instead, high-quality listwise ranking data is required and crucial, calling for further work on building human-annotated listwise data resources.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 28 of its 61 references that are in the map. First posted on arXiv 5 Dec 2023.

Builds on

28 of 61 references.

References · 28

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  10. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  12. 2019Passage Re-ranking with BERTcited by 645
16 more
  1. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 376
  2. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  3. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 307
  4. 2020Overview of the TREC 2019 deep learning trackcited by 286
  5. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 272
  6. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
  7. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  8. 2021Overview of the TREC 2020 deep learning trackcited by 196
  9. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 165
  10. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 115
  11. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 82
  12. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  13. 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
  14. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 70
  15. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  16. 2023Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Modelscited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 2.

14 papers.

Sort:

By area:

  1. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  2. 2024Query Performance Prediction using Relevance Judgments Generated by Large Language Modelscited by 18
  3. 2024LitLLMs, LLMs for Literature Review: Are we there yet?cited by 14
  4. 2024Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languagescited by 8
  5. 2025Scalable In-context Ranking with Generative Modelscited by 8
  6. 2024MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankerscited by 7
  7. 2025The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Modelscited by 3
  8. 2026Temporal Order Matters for Agentic Memory: Segment Trees for Long-Horizon Agentscited by 3
  9. 2025ScaleCall -- Agentic Tool Calling at Scale for Fintech: Challenges, Methods, and Deployment Insightscited by 2
  10. 2024Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Securitycited by 1
  11. 2025Vote-in-Context: Turning VLMs into Zero-Shot Rank Fuserscited by 1
  12. 2025What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Modelscited by 1
2 more
  1. 2024RaCT: Ranking-aware Chain-of-Thought Optimization for LLMscited by 0
  2. 2026Where Root Cause Analysis Fails: A Retrieval-Reranking Decompositioncited by 0