# Query Augmentation by Decoding Semantics from Brain Signals

arXiv: https://arxiv.org/abs/2402.15708 · HTML: https://mlsanity.com/paper/2402.15708/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 60 references that are in the map.
- First posted on arXiv 24 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Language Is Not All You Need: Aligning Perception with Language Models](https://mlsanity.com/paper/2302.14045/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Language Model Evaluation Beyond Perplexity](https://mlsanity.com/paper/2106.00085/) (2021)
- [Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification](https://mlsanity.com/paper/2112.02690/) (2021)
- [UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human language](https://mlsanity.com/paper/2307.05355/) (2023)
- [Query Performance Prediction: From Ad-hoc to Conversational Search](https://mlsanity.com/paper/2305.10923/) (2023)
- [Context Attentive Document Ranking and Query Suggestion](https://mlsanity.com/paper/1906.02329/) (2019)
- [Towards a Better Understanding Human Reading Comprehension with Brain Signals](https://mlsanity.com/paper/2108.01360/) (2021)
- [Natural brain-information interfaces: Recommending information by relevance inferred from human brain signals](https://mlsanity.com/paper/1607.03502/) (2016)
- [Relevance Prediction from Eye-movements Using Semi-interpretable Convolutional Neural Networks](https://mlsanity.com/paper/2001.05152/) (2020)
- [Iterative Relevance Feedback for Answer Passage Retrieval with Passage-level Semantic Match](https://mlsanity.com/paper/1812.08870/) (2018)
- [Brain Topography Adaptive Network for Satisfaction Modeling in Interactive Information Access System](https://mlsanity.com/paper/2208.08097/) (2022)

## Built on by (most cited first, 1 of 1)

- [On the Shape of Brainscores for Large Language Models (LLMs)](https://mlsanity.com/paper/2405.06725/) (2024)

## Abstract (arXiv)

Query augmentation is a crucial technique for refining semantically imprecise queries. Traditionally, query augmentation relies on extracting information from initially retrieved, potentially relevant documents. If the quality of the initially retrieved documents is low, then the effectiveness of query augmentation would be limited as well. We propose Brain-Aug, which enhances a query by incorporating semantic information decoded from brain signals. BrainAug generates the continuation of the original query with a prompt constructed with brain signal information and a ranking-oriented inference approach. Experimental results on fMRI (functional magnetic resonance imaging) datasets show that Brain-Aug produces semantically more accurate queries, leading to improved document ranking performance. Such improvement brought by brain signals is particularly notable for ambiguous queries.