Datasets › WebQuestionsSP

WebQuestionsSP (WebQuestions Semantic Parses Dataset)

Introduced by Wen-tau Yih et al. in The Value of Semantic Parse Labeling for Knowledge Base Question Answering1 Aug 2016 archive 2025-07-28

The WebQuestionsSP dataset is released as part of our ACL-2016 paper “The Value of Semantic Parse Labeling for Knowledge Base Question Answering” [Yih, Richardson, Meek, Chang & Suh, 2016], in which we evaluated the value of gathering semantic parses, vs. answers, for a set of questions that originally comes from WebQuestions [Berant et al., 2013]. The WebQuestionsSP dataset contains full semantic parses in SPARQL queries for 4,737 questions, and “partial” annotations for the remaining 1,073 questions for which a valid parse could not be formulated or where the question itself is bad or needs a descriptive answer. This release also includes an evaluation script and the output of the STAGG semantic parsing system when trained using the full semantic parses. More detail can be found in the document and labeling instructions included in this release, as well as the paper.

Source: WebQuestions Semantic Parses Dataset

Benchmarks archive 2025-07-28

All 3 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Papers archive 2025-07-28

papers with a benchmark row: 12 · with a code link: 10 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (4 of 12 with a benchmark row: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 12 of 12 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 61. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
SPARKLE: Enhancing SPARQL Generation with Direct KG Integration in Decoding 1 1 29 Jun 2024 not harvested
Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation 1 1 26 Jun 2024 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (3 pointer-only for licence)
ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models 1 1 13 Oct 2023 official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 1 violated, 4 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (4 pointer-only for licence)
Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA 1 1 28 Aug 2023 official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (10 pointer-only for licence)
Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family 2 1 14 Mar 2023 not harvested
ReaRev: Adaptive Reasoning for Question Answering over Knowledge Graphs 1 1 24 Oct 2022 not harvested
ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering 1 2 1 Aug 2021 not harvested
Case-based Reasoning for Natural Language Queries over Knowledge Bases 0 1 18 Apr 2021 not harvested
Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals 1 1 11 Jan 2021 not harvested
UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering 1 2 29 Dec 2020 not harvested
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer 57 1 23 Oct 2019 21 ran (of which 0 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 0 violated, 19 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified
The Value of Semantic Parse Labeling for Knowledge Base Question Answering 0 1 1 Aug 2016 not harvested

Dataset loaders archive 2025-07-28

No loader listed in the archive.

Tasks archive 2025-07-28

License archive 2025-07-28

Unknown

Modalities archive 2025-07-28

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • WebQSP-WD
  • WebQuestionsSP

2 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections