Browse State-of-the-Art › Multi-hop Question Answering
Multi-hop Question Answering
92 papers with code · 2 benchmarks · 4 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| ConcurrentQA (1 row) | Multi-hop Dense Passage Retriever (MDR) | Reasoning over Public and Private Data in Retrieval-Based Systems | code | — | Compare |
| MuSiQue-Ans (1 row) | Beam Retrieval | End-to-End Beam Retrieval for Multi-Hop Question Answering | code | Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
4 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 202 · with a code link: 92 · where Syntology ran a sample: 32 (30 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (32 of 202 tagged: 30 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 3: papers 1 to 100 of 202, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
13 Apr 2021 6 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 17 unverified (of 25 harvested samples) · 5 pointer-only (licence)The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify…
-
20 Apr 2019 4 repositories listedWe introduce Multee, a general architecture that can effectively use entailment models for multi-hop QA tasks.
-
17 Aug 2023 3 repositories listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples) · 2 pointer-only (licence)This approach models the multi-hop retrieval process in an end-to-end manner by jointly optimizing an encoder and two classification heads across all hops.
-
2 Aug 2021 3 repositories listed Syntology official (archive's flag): 1 ran · 12 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 16 harvested samples) · 1 pointer-only (licence)Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts.
-
7 Oct 2019 3 repositories listedOur analysis shows the properties of chains that are crucial for high performance: in particular, modeling extraction sequentially is important, as is dealing with each candidate sentence in a context-aware way.
-
23 May 2024 2 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)In order to thrive in hostile and ever-changing natural environments, mammalian brains evolved to store large amounts of knowledge about the world and continually integrate new information while avoiding catastrophic…
-
19 Jan 2024 2 repositories listedHow to edit the knowledge in multi-step reasoning has become the major challenge in the knowledge editing (KE) of large language models (LLMs).
-
24 May 2023 2 repositories listedThe information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option.
-
17 May 2023 2 repositories listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 11 unverified (of 22 harvested samples)We then propose Build-a-Graph Prompting and Algorithmic Prompting, two instruction-based approaches to enhance LLMs in solving natural language graph problems.
-
12 Feb 2023 2 repositories listedTo explain the predicted answers and evaluate the reasoning abilities of models, several studies have utilized underlying reasoning (UR) tasks in multi-hop question answering (QA) datasets.
-
19 Dec 2022 2 repositories listedMulti-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction.
-
1 Jul 2020 2 repositories listedIn a separate line of research, KG embedding methods have been proposed to reduce KG sparsity by performing missing link prediction.
-
15 Apr 2020 2 repositories listed Syntology official (archive's flag): 3 ran · 12 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 1 violated, 7 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 15 harvested samples) · 2 pointer-only (licence)3) a hybrid model that combines heterogeneous information to find the answer.
-
14 May 2019 2 repositories listed Syntology official (archive's flag): 5 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 2 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 1 pointer-only (licence)We propose a new CogQA framework for multi-hop question answering in web-scale documents.
-
25 Sep 2018 2 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 1 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples) · 1 pointer-only (licence)Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers.
-
17 Sep 2018 2 repositories listed Syntology official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)We instead focus on a more challenging multi-hop generative task (NarrativeQA), which requires the model to reason, gather, and synthesize disjoint pieces of information within the context to generate an answer.
-
1 Jun 2025 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 8 where Syntology's instrument failed) · 0 unverified (of 9 harvested samples) · 9 pointer-only (licence)To overcome this limitation, we propose a straightforward approach for dynamically separating and selecting chunks of long context, facilitating a more streamlined input for LLMs.
-
27 May 2025 1 repository listedHowever, the limited context window of LLMs obstructs scaling the amount of external knowledge input, prohibiting further improvement, especially for tasks requiring significant amount of external knowledge.
-
26 May 2025 1 repository listedRecent advances in retrieval-augmented generation (RAG) furnish large language models (LLMs) with iterative retrievals of relevant information to handle complex multi-hop questions.
-
26 May 2025 1 repository listedIn the pre-training stage, we introduce the Retrieval Augmented Mask Prediction (RAMP) task, where the model learns to leverage search tools to fill masked spans on a large number of pre-training data, thus acquiring…
-
25 May 2025 1 repository listedRecent work explores iterative retrieval to address complex problems.
-
25 May 2025 1 repository listedBy utilizing reinforcement learning, SituatedThinker incentivizes deliberate reasoning with the real world to acquire information and feedback, allowing LLMs to surpass their knowledge boundaries and enhance reasoning.
-
21 May 2025 1 repository listedMulti-Hop Question Answering (MHQA) is crucial for evaluating the model's capability to integrate information from diverse sources.
-
16 May 2025 1 repository listedOur study reveals interesting findings as follows: 1) Encoder-decoder models, such as the ones in the Flan-T5 family, generally outperform causal decoder-only LMs in MHQA tasks, despite being significantly smaller in…
-
14 May 2025 1 repository listedIn many real-world settings, machine learning models and interactive systems have access to both structured knowledge, e. g., knowledge graphs or tables, and unstructured content, e. g., natural language documents.
-
28 Apr 2025 1 repository listedRetrieval-augmented generation (RAG) systems face significant challenges in multi-hop question answering (MHQA), where complex queries require synthesizing information across multiple document chunks.
-
7 Apr 2025 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)Retrieval-Augmented Generation (RAG) systems often struggle to handle multi-hop question-answering tasks accurately due to irrelevant context retrieval and limited complex reasoning capabilities.
-
13 Mar 2025 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples)Graph-based Retrieval-Augmented Generation (RAG) methods have significantly enhanced the performance of large language models (LLMs) in domain-specific tasks.
-
25 Feb 2025 1 repository listedIn LevelRAG, the high-level searcher orchestrates the retrieval logic, while the low-level searchers (sparse, web, and dense) refine the queries for optimal retrieval.
-
22 Dec 2024 1 repository listedTo bridge this gap, we introduce MINTQA (Multi-hop Question Answering on New and Tail Knowledge), a comprehensive benchmark to evaluate LLMs' capabilities in multi-hop reasoning across four critical dimensions: question…
-
18 Dec 2024 1 repository listed
-
13 Dec 2024 1 repository listed
-
7 Dec 2024 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 3 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
16 Oct 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 1 pointer-only (licence)
-
6 Oct 2024 1 repository listed
-
21 Sep 2024 1 repository listed
-
19 Sep 2024 1 repository listed
-
26 Aug 2024 1 repository listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
21 Aug 2024 1 repository listed
-
20 Aug 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
8 Aug 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
12 Jul 2024 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 3 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples) · 10 pointer-only (licence)
-
5 Jul 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 3 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 5 pointer-only (licence)
-
10 Jun 2024 1 repository listed Syntology official (archive's flag): 5 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 2 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
3 Jun 2024 1 repository listed
-
3 May 2024 1 repository listed
-
28 Mar 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples) · 8 pointer-only (licence)
-
19 Mar 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
18 Feb 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 2 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
18 Feb 2024 1 repository listed
-
17 Feb 2024 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 1 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples) · 13 pointer-only (licence)
-
8 Feb 2024 1 repository listed
-
23 Dec 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
4 Dec 2023 1 repository listed
-
29 Nov 2023 1 repository listed
-
7 Nov 2023 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified (of 13 harvested samples) · 13 pointer-only (licence)
-
20 Oct 2023 1 repository listed
-
20 Sep 2023 1 repository listed
-
7 Aug 2023 1 repository listed
-
24 May 2023 1 repository listed
-
28 Apr 2023 1 repository listed
-
25 Apr 2023 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples)
-
26 Mar 2023 1 repository listed
-
31 Jan 2023 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 1 pointer-only (licence)
-
16 Dec 2022 1 repository listed
-
2 Dec 2022 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified (of 17 harvested samples)
-
22 Aug 2022 1 repository listed
-
1 Jul 2022 1 repository listed
-
1 Jun 2022 1 repository listed
-
14 Mar 2022 1 repository listed
-
14 Feb 2022 1 repository listed
-
22 Nov 2021 1 repository listed
-
25 Oct 2021 1 repository listed
-
25 Jul 2021 1 repository listed
-
15 Jun 2021 1 repository listed
-
25 May 2021 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 2 samples that ran constructed an object rather than computing a result (of 2 harvested samples) · 2 pointer-only (licence)
-
15 Apr 2021 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
13 Apr 2021 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 5 harvested samples) · 5 pointer-only (licence)
-
31 Jan 2021 1 repository listed
-
9 Nov 2020 1 repository listed
-
2 Nov 2020 1 repository listed Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 1 pointer-only (licence)
-
29 Oct 2020 1 repository listed
-
23 Oct 2020 1 repository listed
-
7 Oct 2020 1 repository listed
-
4 May 2020 1 repository listed
-
2 May 2020 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 3 unverified (of 3 harvested samples)
-
1 May 2020 1 repository listed
-
9 Nov 2019 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 1 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 5 pointer-only (licence)
-
4 Nov 2019 1 repository listed
-
25 Oct 2019 1 repository listed
-
19 Sep 2019 1 repository listed
-
17 Jun 2019 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples)
-
Memento: Note-Taking for Your Future Self25 Jun 2025 0 repositories listed
-
Inference Scaled GraphRAG: Improving Multi Hop Question Answering on Knowledge Graphs24 Jun 2025 0 repositories listed
-
KunLunBaizeRAG: Reinforcement Learning Driven Inference Performance Leap for Large Language Models24 Jun 2025 0 repositories listed
-
Mental Health Equity in LLMs: Leveraging Multi-Hop Question Answering to Detect Amplified and Silenced Perspectives22 Jun 2025 0 repositories listed
-
A Graph-Retrieval-Augmented Generation Framework Enhances Decision-Making in the Circular Economy1 Jun 2025 0 repositories listed
-
EvolveSearch: An Iterative Self-Evolving Search Agent28 May 2025 0 repositories listed
-
Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA22 May 2025 0 repositories listed
-
BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering17 May 2025 0 repositories listed
Syntology lines on 36 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections