# Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systems

arXiv: https://arxiv.org/abs/2506.00585 · HTML: https://mlsanity.com/paper/2506.00585/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 31 May 2025.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [Key-Value Retrieval Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1705.05414/) (2017)
- [RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder](https://mlsanity.com/paper/2205.12035/) (2022)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [Residual Energy-Based Models for Text Generation](https://mlsanity.com/paper/2004.11714/) (2020)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [Pre-Training Transformers as Energy-Based Cloze Models](https://mlsanity.com/paper/2012.08561/) (2020)
- [Hindsight: Posterior-guided training of retrievers for improved open-ended generation](https://mlsanity.com/paper/2110.07752/) (2021)
- [Zero-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2008.12918/) (2020)
- [Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation](https://mlsanity.com/paper/2112.07924/) (2021)
- [Leveraging LLMs for Unsupervised Dense Retriever Ranking](https://mlsanity.com/paper/2402.04853/) (2024)
- [Global Autoregressive Models for Data-Efficient Sequence Learning](https://mlsanity.com/paper/1909.07063/) (2019)
- [Semi-Supervised Knowledge-Grounded Pre-training for Task-Oriented Dialog Systems](https://mlsanity.com/paper/2210.08873/) (2022)
- [Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models](https://mlsanity.com/paper/2101.06829/) (2021)
- [RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling](https://mlsanity.com/paper/2310.10567/) (2023)
- [Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models](https://mlsanity.com/paper/2005.14001/) (2020)
- [Learning neural trans-dimensional random field language models with noise-contrastive estimation](https://mlsanity.com/paper/1710.10739/) (2017)
- [Knowledge-Retrieval Task-Oriented Dialog Systems with Semi-Supervision](https://mlsanity.com/paper/2305.13199/) (2023)
- [A Challenge on Semi-Supervised and Reinforced Task-Oriented Dialog Systems](https://mlsanity.com/paper/2207.02657/) (2022)
- [Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation](https://mlsanity.com/paper/1807.00993/) (2018)
- [Language modeling with Neural trans-dimensional random fields](https://mlsanity.com/paper/1707.07240/) (2017)
- [Information Extraction and Human-Robot Dialogue towards Real-life Tasks: A Baseline Study with the MobileCS Dataset](https://mlsanity.com/paper/2209.13464/) (2022)

## Abstract (arXiv)

A retriever, which retrieves relevant knowledge pieces from a knowledge base given a context, is an important component in many natural language processing (NLP) tasks. Retrievers have been introduced in knowledge-grounded dialog systems to improve knowledge acquisition. In knowledge-grounded dialog systems, when conditioning on a given context, there may be multiple relevant and correlated knowledge pieces. However, knowledge pieces are usually assumed to be conditionally independent in current retriever models. To address this issue, we propose Entriever, an energy-based retriever. Entriever directly models the candidate retrieval results as a whole instead of modeling the knowledge pieces separately, with the relevance score defined by an energy function. We explore various architectures of energy functions and different training methods for Entriever, and show that Entriever substantially outperforms the strong cross-encoder baseline in knowledge retrieval tasks. Furthermore, we show that in semi-supervised training of knowledge-grounded dialog systems, Entriever enables effective scoring of retrieved knowledge pieces and significantly improves end-to-end performance of dialog systems.