# Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks

arXiv: https://arxiv.org/abs/2306.04009 · HTML: https://mlsanity.com/paper/2306.04009/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 1 of its 35 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 1 of 1)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)

## Built on by (most cited first, 2 of 2)

- [Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation](https://mlsanity.com/paper/2402.03268/) (2024)
- [BioHopR: A Benchmark for Multi-Hop, Multi-Answer Reasoning in Biomedical Domain](https://mlsanity.com/paper/2505.22240/) (2025)

## Abstract (arXiv)

Despite readily memorizing world knowledge about entities, pre-trained language models (LMs) struggle to compose together two or more facts to perform multi-hop reasoning in question-answering tasks. In this work, we propose techniques that improve upon this limitation by relying on random walks over structured knowledge graphs. Specifically, we use soft prompts to guide LMs to chain together their encoded knowledge by learning to map multi-hop questions to random walk paths that lead to the answer. Applying our methods on two T5 LMs shows substantial improvements over standard tuning approaches in answering questions that require 2-hop reasoning.