Browse State-of-the-Art › StrategyQA
StrategyQA
20 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
StrategyQA aims to measure the ability of models to answer questions that require multi-step implicit reasoning.
Source: BIG-bench
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 40 · with a code link: 20 · where Syntology ran a sample: 11 (10 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (11 of 40 tagged: 10 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 40 of 40, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
5 Apr 2022 7 repositories listed Syntology 32 ran (of which 16 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified (of 37 harvested samples)To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
-
12 Aug 2024 3 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples)This paper introduces rStar, a self-play mutual reasoning approach that significantly improves reasoning capabilities of small language models (SLMs) without fine-tuning or superior models.
-
30 Sep 2023 3 repositories listedTo improve planning with LLMs, we propose an agentic architecture, the Modular Agentic Planner (MAP), in which planning is accomplished via the recurrent interaction of the specialized modules mentioned above, each…
-
21 Mar 2022 3 repositories listed Syntology 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample)Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks.
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
29 Mar 2022 2 repositories listed Syntology 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 4 pointer-only (licence)We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget.
-
4 Mar 2025 1 repository listedLarge Language Models (LLMs) are increasingly being used in real-world applications.
-
26 Feb 2025 1 repository listedOur results show that voting protocols improve performance by 13.
-
9 Dec 2024 1 repository listedChain of Thought (CoT) was introduced in recent research as a method for improving step-by-step reasoning in Large Language Models.
-
7 Oct 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)Furthermore, we demonstrate that training a verifier on valid rationales significantly improves its ability to distinguish valid and flawed rationale.
-
23 May 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)In MoE, each token in the input sequence activates a different subset of experts determined by a routing mechanism.
-
3 Mar 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)In this work, we seek a novel KGQA dataset that supports commonsense reasoning and focuses on long-tail entities (e. g., non-mainstream and recent entities) where LLMs frequently hallucinate, and thus create the need…
-
21 Feb 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples) · 10 pointer-only (licence)We propose a straightforward approach called Distillation Contrastive Decoding (DCD) to enhance the reasoning capabilities of Large Language Models (LLMs) during inference.
-
19 Jan 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 3 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 8 pointer-only (licence)Self-consistency (SC) has been a widely used decoding strategy for chain-of-thought reasoning.
-
6 Nov 2023 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples) · 10 pointer-only (licence)Results on five difficult question-answering datasets StrategyQA, QuaRel, OpenBookQA, NumerSense and QASC show that not only does MaRio improve task accuracy, but it also improves the self-rationalization quality of…
-
2 Aug 2023 1 repository listedWe equip a smaller Language Model to generalise to answering challenging compositional questions that have not been seen in training.
-
28 May 2023 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 1 pointer-only (licence)Large Language Models (LLMs) have shown promising performance in knowledge-intensive reasoning tasks that require a compound understanding of knowledge.
-
19 Dec 2022 1 repository listedThis paper proposes a question-answering system that can answer questions whose supporting evidence is spread over multiple (potentially long) documents.
-
1 Dec 2022 1 repository listedIn this work, we propose an alternative reasoning scheme, Socratic CoT, that learns a decomposition of the original problem into a sequence of subproblems and uses it to guide the intermediate reasoning steps.
-
6 Jan 2021 1 repository listedA key limitation in current datasets for multi-hop reasoning is that the required steps for answering the question are mentioned in it explicitly.
-
Fusing Bidirectional Chains of Thought and Reward Mechanisms A Method for Enhancing Question-Answering Capabilities of Large Language Models for Chinese Intangible Cultural Heritage13 May 2025 0 repositories listedThe rapid development of large language models (LLMs) has provided significant support and opportunities for the advancement of domain-specific LLMs.
-
Rule-Guided Feedback: Enhancing Reasoning by Enforcing Rule Adherence in Large Language Models14 Mar 2025 0 repositories listedIn this paper, we introduce Rule-Guided Feedback (RGF), a framework designed to enhance Large Language Model (LLM) performance through structured rule adherence and strategic information seeking.
-
Unraveling Indirect In-Context Learning Using Influence Functions1 Jan 2025 0 repositories listedThis work introduces a novel paradigm for generalized In-Context Learning (ICL), termed Indirect In-Context Learning.
-
Dialectical Behavior Therapy Approach to LLM Prompting10 Oct 2024 0 repositories listedLarge language models demonstrated state-of-the-art results on various reasoning tasks when applying the chain-of-thought (CoT) prompting technique.
-
A Looming Replication Crisis in Evaluating Behavior in Language Models? Evidence and Solutions30 Sep 2024 0 repositories listedIn an era where large language models (LLMs) are increasingly integrated into a wide range of everyday applications, research into these models' behavior has surged.
-
Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoning25 Sep 2024 0 repositories listedLarge Language Models (LLMs) have revolutionized natural language processing, yet they struggle with inconsistent reasoning, particularly in novel domains and complex logical sequences.
-
Meta-prompting Optimized Retrieval-augmented Generation4 Jul 2024 0 repositories listedRetrieval-augmented generation resorts to content retrieved from external sources in order to leverage the performance of large language models in downstream tasks.
-
Question-Analysis Prompting Improves LLM Performance in Reasoning Tasks4 Jul 2024 0 repositories listedAlthough LLMs have the potential to transform many fields, they still underperform humans in reasoning tasks.
-
Advancing Process Verification for Large Language Models via Tree-Based Preference Learning29 Jun 2024 0 repositories listedLarge Language Models (LLMs) have demonstrated remarkable potential in handling complex reasoning tasks by generating step-by-step rationales.Some methods have proven effective in boosting accuracy by introducing extra…
-
Improving Attributed Text Generation of Large Language Models via Preference Learning27 Mar 2024 0 repositories listedLarge language models have been widely adopted in natural language processing, yet they face the challenge of generating unreliable content.
-
Towards Uncertainty-Aware Language Agent25 Jan 2024 0 repositories listed
-
IAG: Induction-Augmented Generation Framework for Answering Reasoning Questions30 Nov 2023 0 repositories listed
-
14 Nov 2023 0 repositories listed
-
Large Language Models Are Also Good Prototypical Commonsense Reasoners22 Sep 2023 0 repositories listed
-
Answering Unseen Questions With Smaller Language Models Using Rationale Generation and Dense Retrieval9 Aug 2023 0 repositories listed
-
Deduction under Perturbed Evidence: Probing Student Simulation Capabilities of Large Language Models23 May 2023 0 repositories listed
-
Hint of Thought prompting: an explainable and zero-shot approach to reasoning tasks with LLMs19 May 2023 0 repositories listed
-
Self-Evaluation Guided Beam Search for Reasoning1 May 2023 0 repositories listed
-
Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts30 Oct 2022 0 repositories listed
-
Better Retrieval May Not Lead to Better Question Answering7 May 2022 0 repositories listed
Syntology lines on 11 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections