Retrieving a Set, Not Independent Passages: Set-Level Compatibility Learning for Efficient Set Exploration

Multi-hop question answering requires retrieving multiple evidence passages whose usefulness often depends on one another. Conventional retrievers either rank passages independently or construct evidence sequentially through locally supervised next-passage decisions. Sequential conditioning captures some cross-passage dependencies, but its local extension scores do not provide a common criterion for comparing complete evidence sets of different compositions and sizes. Existing multi-hop retrievers therefore avoid directly learning a query--set compatibility function over complete evidence sets, as the exponentially large set space is daunting to cover during learning and impractical to search at inference time. To overcome these challenges, we formulate multi-hop retrieval by directly ranking candidate evidence sets with an energy-based query--set compatibility score sθ(q,S), learned from informative contrasts without exhaustive coverage or normalization of the combinatorial set space. Given a gold evidence set, we automatically generate contrasts by adding, removing, or replacing passages, yielding rich set-level supervision without additional annotation. To make search over this combinatorial space practical at inference time, we introduce a novel retrieve-and-rerank framework over the set space: ParaSet, a lightweight scorer over precomputed passage representations for efficient set exploration, and SetCE, an expressive cross-encoder for set reranking. Our experiments show that set-level retrieval consistently provides a complementary signal to passage-level relevance, becoming relatively more effective when more hops are required to reach evidence from the query. Motivated by this complementarity, combining the two signals further improves downstream QA performance, outperforming both deeper passage-level retrieval and an ensemble of distinct passage-level retrievers.