Reliable Academic Conference Question Answering: A Study Based on Large Language Model
Not on the map. Why?
Abstract (arXiv)
As the development of academic conferences fosters global scholarly communication, researchers consistently need to obtain accurate and up-to-date information about academic conferences. Since the information is scattered, using an intelligent question-answering system to efficiently handle researchers' queries and ensure awareness of the latest advancements is necessary. Recently, Large Language Models (LLMs) have demonstrated impressive capabilities in question answering, and have been enhanced by retrieving external knowledge to deal with outdated knowledge. However, these methods fail to work due to the lack of the latest conference knowledge. To address this challenge, we develop the ConferenceQA dataset, consisting of seven diverse academic conferences. Specifically, for each conference, we first organize academic conference data in a tree-structured format through a semi-automated method. Then we annotate question-answer pairs and classify the pairs into four different types to better distinguish their difficulty. With the constructed dataset, we further propose a novel method STAR (STructure-Aware Retrieval) to improve the question-answering abilities of LLMs, leveraging inherent structural information during the retrieval process. Experimental results on the ConferenceQA dataset show the effectiveness of our retrieval method. The dataset and code are available at https://github.com/zjukg/ConferenceQA.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 50 references that are in the map. First posted on arXiv 19 Oct 2023.
Builds on
35 of 50 references.
References · 35
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
23 more
- 2023Mistral 7Bcited by 3,164
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
- 2021Improving language models by retrieving from trillions of tokenscited by 1,293
- 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
- 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
- 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
- 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
- 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
- 2023Active Retrieval Augmented Generationcited by 516
- 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
- 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
- 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
- 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
- 2023ChatDB: Augmenting LLMs with Databases as Their Symbolic Memorycited by 140
- 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
- 2023Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answeringcited by 65
- 2023RET-LLM: Towards a General Read-Write Memory for Large Language Modelscited by 64
- 2023The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Studycited by 17
- 2022End-to-End Table Question Answering via Retrieval-Augmented Generationcited by 14
Built on by
Citing papers per year: 2025: 1.
1 paper.