# Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-based Reranking

arXiv: https://arxiv.org/abs/2112.01206 · HTML: https://mlsanity.com/paper/2112.01206/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features](https://mlsanity.com/paper/1703.02507/) (2017)
- [A Deep Look into Neural Ranking Models for Information Retrieval](https://mlsanity.com/paper/1903.06902/) (2019)
- [Content-Based Citation Recommendation](https://mlsanity.com/paper/1802.08301/) (2018)
- [Citation Recommendation: Approaches and Datasets](https://mlsanity.com/paper/2002.06961/) (2020)
- [A Context-Aware Citation Recommendation Model with BERT and Graph Convolutional Networks](https://mlsanity.com/paper/1903.06464/) (2019)
- [HybridCite: A Hybrid Model for Context-Aware Citation Recommendation](https://mlsanity.com/paper/2002.06406/) (2020)
- [Embedding-based Scientific Literature Discovery in a Text Editor Application](https://mlsanity.com/paper/2005.04961/) (2020)

## Built on by (most cited first, 19 of 19)

- [LitSearch: A Retrieval Benchmark for Scientific Literature Search](https://mlsanity.com/paper/2407.18940/) (2024)
- [CiteME: Can Language Models Accurately Cite Scientific Claims?](https://mlsanity.com/paper/2407.12861/) (2024)
- [When Large Language Models Meet Citation: A Survey](https://mlsanity.com/paper/2309.09727/) (2023)
- [CiteBART: Learning to Generate Citations for Local Citation Recommendation](https://mlsanity.com/paper/2412.17534/) (2024)
- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)
- [Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attribution](https://mlsanity.com/paper/2509.21557/) (2025)
- [SciLit: A Platform for Joint Scientific Literature Discovery, Summarization and Citation Generation](https://mlsanity.com/paper/2306.03535/) (2023)
- [Large-scale Evaluation of Transformer-based Article Encoders on the Task of Citation Recommendation](https://mlsanity.com/paper/2209.05452/) (2022)
- [Referral Augmentation for Zero-Shot Information Retrieval](https://mlsanity.com/paper/2305.15098/) (2023)
- [ILCiteR: Evidence-grounded Interpretable Local Citation Recommendation](https://mlsanity.com/paper/2403.08737/) (2024)
- [SymTax: Symbiotic Relationship and Taxonomy Fusion for Effective Citation Recommendation](https://mlsanity.com/paper/2406.01606/) (2024)
- [SCIRGC: Multi-Granularity Citation Recommendation and Citation Sentence Preference Alignment](https://mlsanity.com/paper/2505.20103/) (2025)
- [AgentExpt: Automating AI Experiment Design with LLM-based Resource Retrieval Agent](https://mlsanity.com/paper/2511.04921/) (2025)
- [Paragraph-level Citation Recommendation based on Topic Sentences as Queries](https://mlsanity.com/paper/2305.12190/) (2023)
- [MemSum-DQA: Adapting An Efficient Long Document Extractive Summarizer for Document Question Answering](https://mlsanity.com/paper/2310.06436/) (2023)
- [Public Profile Matters: A Scalable Integrated Approach to Recommend Citations in the Wild](https://mlsanity.com/paper/2603.17361/) (2026)
- [Talk2Ref: A Dataset for Reference Prediction from Scientific Talks](https://mlsanity.com/paper/2510.24478/) (2025)
- [ReCite: Agentic Reasoning for Faithful Citation](https://mlsanity.com/paper/2609.09156/) (2026)
- [Local Citation Recommendation Without Graph Signals](https://mlsanity.com/paper/2610.15832/) (2026)

## Abstract (arXiv)

The goal of local citation recommendation is to recommend a missing reference from the local citation context and optionally also from the global context. To balance the tradeoff between speed and accuracy of citation recommendation in the context of a large-scale paper database, a viable approach is to first prefetch a limited number of relevant documents using efficient ranking methods and then to perform a fine-grained reranking using more sophisticated models. In that vein, BM25 has been found to be a tough-to-beat approach to prefetching, which is why recent work has focused mainly on the reranking step. Even so, we explore prefetching with nearest neighbor search among text embeddings constructed by a hierarchical attention network. When coupled with a SciBERT reranker fine-tuned on local citation recommendation tasks, our hierarchical Attention encoder (HAtten) achieves high prefetch recall for a given number of candidates to be reranked. Consequently, our reranker requires fewer prefetch candidates to rerank, yet still achieves state-of-the-art performance on various local citation recommendation datasets such as ACL-200, FullTextPeerRead, RefSeer, and arXiv.