# Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction

arXiv: https://arxiv.org/abs/2311.05922 · HTML: https://mlsanity.com/paper/2311.05922/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 12 of its 40 references that are in the map.
- First posted on arXiv 10 Nov 2023.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 12 of 12)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering](https://mlsanity.com/paper/2212.10375/) (2022)
- [Instruction Induction: From Few Examples to Natural Language Task Descriptions](https://mlsanity.com/paper/2205.10782/) (2022)
- [GPT-RE: In-context Learning for Relation Extraction using Large Language Models](https://mlsanity.com/paper/2305.02105/) (2023)
- [Fairness-guided Few-shot Prompting for Large Language Models](https://mlsanity.com/paper/2303.13217/) (2023)
- [Relation Extraction : A Survey](https://mlsanity.com/paper/1712.05191/) (2017)
- [Function-words Enhanced Attention Networks for Few-Shot Inverse Relation Classification](https://mlsanity.com/paper/2204.12111/) (2022)

## Built on by (most cited first, 19 of 19)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [C-ICL: Contrastive In-context Learning for Information Extraction](https://mlsanity.com/paper/2402.11254/) (2024)
- [Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors](https://mlsanity.com/paper/2404.17807/) (2024)
- [Multi-objective Large Language Model Alignment with Hierarchical Experts](https://mlsanity.com/paper/2505.20925/) (2025)
- [Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Construction](https://mlsanity.com/paper/2312.03022/) (2023)
- [Collaborative Evolving Strategy for Automatic Data-Centric Development](https://mlsanity.com/paper/2407.18690/) (2024)
- [Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogs](https://mlsanity.com/paper/2510.23941/) (2025)
- [PGA-SciRE: Harnessing LLM on Data Augmentation for Enhancing Scientific Relation Extraction](https://mlsanity.com/paper/2405.20787/) (2024)
- [Level-Navi Agent: A Framework and benchmark for Chinese Web Search Agents](https://mlsanity.com/paper/2502.15690/) (2024)
- [AMR-RE: Abstract Meaning Representations for Retrieval-Based In-Context Learning in Relation Extraction](https://mlsanity.com/paper/2406.10432/) (2024)
- [MMAPG: A Training-Free Framework for Multimodal Multi-hop Question Answering via Adaptive Planning Graphs](https://mlsanity.com/paper/2508.16051/) (2025)
- [Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning Applied to Few-Shot Relation Extraction](https://mlsanity.com/paper/2601.20803/) (2026)
- [DiVA-DocRE: A Discriminative and Voice-Aware Paradigm for Document-Level Relation Extraction](https://mlsanity.com/paper/2409.13717/) (2024)
- [Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval](https://mlsanity.com/paper/2506.21222/) (2025)
- [Error-Aware Curriculum Learning for Biomedical Relation Classification](https://mlsanity.com/paper/2507.14374/) (2025)
- [The Answer Lies Within: Self-Derived Rewards Enable Explainable Relation Extraction](https://mlsanity.com/paper/2510.06198/) (2025)
- [Relation as a Prior: A Novel Paradigm for LLM-based Document-level Relation Extraction](https://mlsanity.com/paper/2511.08143/) (2025)
- [LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction](https://mlsanity.com/paper/2606.29407/) (2026)
- [Two-Stage Prompt Optimization for Few-Shot Relation Extraction: From Reasoning-Guided Search to Gradient-Guided Refinement](https://mlsanity.com/paper/2606.29639/) (2026)

## Abstract (arXiv)

Few-shot relation extraction involves identifying the type of relationship between two specific entities within a text, using a limited number of annotated samples. A variety of solutions to this problem have emerged by applying meta-learning and neural graph techniques which typically necessitate a training process for adaptation. Recently, the strategy of in-context learning has been demonstrating notable results without the need of training. Few studies have already utilized in-context learning for zero-shot information extraction. Unfortunately, the evidence for inference is either not considered or implicitly modeled during the construction of chain-of-thought prompts. In this paper, we propose a novel approach for few-shot relation extraction using large language models, named CoT-ER, chain-of-thought with explicit evidence reasoning. In particular, CoT-ER first induces large language models to generate evidences using task-specific and concept-level knowledge. Then these evidences are explicitly incorporated into chain-of-thought prompting for relation extraction. Experimental results demonstrate that our CoT-ER approach (with 0% training data) achieves competitive performance compared to the fully-supervised (with 100% training data) state-of-the-art approach on the FewRel1.0 and FewRel2.0 datasets.