Browse State-of-the-Art › Semantic Parsing
Semantic Parsing
414 papers with code · 20 benchmarks · 44 datasets archive 2025-07-28
Semantic Parsing is the task of transducing natural language utterances into formal meaning representations. The target meaning representations can be defined according to a wide variety of formalisms. This include linguistically-motivated semantic representations that are designed to capture the meaning of any sentence such as λ-calculus or the abstract meaning representations. Alternatively, for more task-driven approaches to Semantic Parsing, it is common for meaning representations to represent executable programs such as SQL queries, robotic commands, smart phone instructions, and even general-purpose programming languages like Python and Java.
Source: Tranx: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
20 leaderboard tables shown for this task, 20 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 20 until expanded.
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
44 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 44 until expanded.
Subtasks archive 2025-07-28
7 subtasks in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 1,202 · with a code link: 414 · where Syntology ran a sample: 96 (80 with a run with no instrument failure, 16 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (96 of 1,202 tagged: 80 with a run with no instrument failure, 16 where every run was a failure of Syntology's instrument)
Page 1 of 13: papers 1 to 100 of 1,202, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
2 Dec 2016 110 repositories listed Syntology 103 ran (of which 45 constructed an object rather than computing a result; 74 with no instrument failure: 0 honoured, 0 violated, 74 with no contract checked; 29 where Syntology's instrument failed) · 61 unverified (of 164 harvested samples) · 92 pointer-only (licence)Point cloud is an important type of geometric data structure.
-
23 Oct 2019 57 repositories listed Syntology 21 ran (of which 0 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 0 violated, 19 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified (of 31 harvested samples)Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
-
29 Jun 2017 10 repositories listedWe present a new, efficient frame-semantic parser that labels semantic arguments to FrameNet predicates.
-
5 Apr 2020 8 repositories listed Syntology community repositories only · 14 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 14 harvested samples)In this paper, we present TAPAS, an approach to question answering over tables without generating logical forms.
-
20 Jun 2018 7 repositories listedSemantic parsing is the task of transducing natural language (NL) utterances into formal meaning representations (MRs), commonly represented as tree structures.
-
24 Sep 2018 6 repositories listed Syntology official (archive's flag): 3 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 3 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 3 pointer-only (licence)We define a new complex and cross-domain semantic parsing and text-to-SQL task where different complex SQL queries and databases appear in train and test sets.
-
20 Jun 2018 6 repositories listed Syntology official (archive's flag): 9 ran · 13 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 13 harvested samples) · 4 pointer-only (licence)Though designed for decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic parsing task in the single-task setting.
-
6 Apr 2017 6 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)We consider the problem of parsing natural language descriptions into source code written in a general-purpose programming language like Python.
-
16 Oct 2019 5 repositories listedWe present a simple methods to leverage the table content for the BERT-based model to solve the text-to-SQL problem.
-
4 Feb 2019 5 repositories listed Syntology official (archive's flag): 3 ran · 21 ran (of which 1 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 2 violated, 1 with no contract checked; 16 where Syntology's instrument failed) · 15 unverified (of 36 harvested samples) · 33 pointer-only (licence)We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to achieve human performance in WikiSQL dataset.
-
6 Jan 2016 5 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 7 harvested samples) · 1 pointer-only (licence)Semantic parsing aims at mapping natural language to machine interpretable meaning representations.
-
6 Oct 2022 4 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e. g., SQL…
-
31 May 2022 4 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)Through these explorations, we show that PI can be a promising direction for conditioning language models, especially in scenarios with long and fixed prompts.
-
16 Jul 2021 4 repositories listedTAPEX addresses the data scarcity challenge via guiding the language model to mimic a SQL executor on the diverse, large-scale and high-quality synthetic corpus.
-
16 Apr 2020 4 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples)In this technical report, we present two novel datasets for image scene understanding.
-
31 Jan 2020 4 repositories listedUnderstanding natural language questions entails the ability to break down a question into the requisite steps for computing its answer.
-
10 Nov 2019 4 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 4 pointer-only (licence)The generalization challenge lies in (a) encoding the database relations in an accessible way for the semantic parser, and (b) modeling alignment between database columns and their mentions in a given query.
-
5 Jun 2019 4 repositories listed Syntology official (archive's flag): 1 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 3 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 3 pointer-only (licence)The best model obtains an exact match accuracy of 20.
-
5 Oct 2018 4 repositories listed Syntology official (archive's flag): 5 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 2 pointer-only (licence)We present TRANX, a transition-based neural semantic parser that maps natural language (NL) utterances into formal meaning representations (MRs).
-
6 Jul 2018 4 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified (of 15 harvested samples)We present Memory Augmented Policy Optimization (MAPO), a simple and novel way to leverage a memory buffer of promising trajectories to reduce the variance of policy gradient estimate.
-
3 Aug 2015 4 repositories listedTwo important aspects of semantic parsing for question answering are the breadth of the knowledge source and the depth of logical compositionality.
-
6 Mar 2023 3 repositories listedHowever, the implementation of ICL is sophisticated due to the diverse retrieval and inference methods involved, as well as the varying pre-processing requirements for different models, datasets, and tasks.
-
10 Sep 2021 3 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 3 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)Large pre-trained language models for textual data have an unconstrained output space; at each decoding step, they can produce any of 10, 000s of sub-word tokens.
-
18 Dec 2020 3 repositories listedMost recently, there has been significant interest in learning contextual representations for various NLP tasks, by leveraging large scale text corpora to train large neural language models with self-supervised learning…
-
9 Oct 2020 3 repositories listed Syntology official (archive's flag): 13 ran · 13 ran (of which 2 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 4 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 13 unverified (of 26 harvested samples) · 26 pointer-only (licence)To demonstrate the generality of AutoQA, we also apply it to the Overnight dataset.
-
16 Jan 2020 3 repositories listedThe key concept is to cover the space of possible compound queries on the database with a large number of in-domain questions synthesized with the help of a corpus of generic query templates.
-
20 Dec 2019 3 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)We present a large and realistic natural language question answering dataset that is constructed according to this method, and we use it to analyze the compositional generalization ability of three machine learning…
-
1 Mar 2019 3 repositories listedWe introduce a new English reading comprehension benchmark, DROP, which requires Discrete Reasoning Over the content of Paragraphs.
-
25 Feb 2018 3 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)We present new data and semantic parsing methods for the problem of mapping English sentences to Bash commands (NL2Bash).
-
25 Apr 2017 3 repositories listedOur goal is to learn a semantic parser that maps natural language utterances into executable programs when only indirect supervision is available: examples are labeled with the correct execution result, but not the…
-
8 Jun 2016 3 repositories listed
-
10 Jun 2024 2 repositories listed
-
15 Apr 2024 2 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 6 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified (of 15 harvested samples) · 15 pointer-only (licence)
-
16 Jun 2023 2 repositories listed
-
6 Jun 2023 2 repositories listed
-
30 May 2023 2 repositories listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 1 pointer-only (licence)
-
14 May 2023 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
14 Mar 2023 2 repositories listed
-
15 Feb 2023 2 repositories listed
-
31 Jan 2023 2 repositories listed
-
3 Jan 2023 2 repositories listed
-
14 Nov 2022 2 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 1 pointer-only (licence)
-
14 Dec 2021 2 repositories listed
-
22 Sep 2021 2 repositories listed
-
7 Aug 2021 2 repositories listed
-
14 Jul 2021 2 repositories listed
-
6 May 2021 2 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 6 unverified (of 14 harvested samples)
-
15 Apr 2021 2 repositories listed
-
5 Jan 2021 2 repositories listed
-
23 Dec 2020 2 repositories listed
-
9 Nov 2020 2 repositories listed
-
2 Nov 2020 2 repositories listed
-
21 Oct 2020 2 repositories listed Syntology official (archive's flag): 1 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
-
8 Jul 2020 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 9 unverified (of 10 harvested samples)
-
6 Jul 2020 2 repositories listed Syntology official (archive's flag): 2 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 14 harvested samples)
-
14 Mar 2020 2 repositories listed
-
22 Nov 2019 2 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
11 Oct 2019 2 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
-
29 Sep 2019 2 repositories listed
-
1 Jun 2019 2 repositories listed
-
26 Apr 2019 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)
-
6 Mar 2019 2 repositories listed Syntology official (archive's flag): 1 ran · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples)
-
11 Oct 2018 2 repositories listed
-
12 May 2018 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)
-
17 Apr 2018 2 repositories listed
-
19 Mar 2018 2 repositories listed
-
23 Feb 2018 2 repositories listed
-
19 Oct 2017 2 repositories listed Syntology official (archive's flag): 4 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
25 Jul 2017 2 repositories listed
-
1 Jun 2017 2 repositories listed
-
28 May 2017 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
19 Apr 2017 2 repositories listed
-
20 Jun 2016 2 repositories listed
-
29 Jun 2025 1 repository listed
-
5 Apr 2025 1 repository listed
-
4 Apr 2025 1 repository listed
-
25 Feb 2025 1 repository listed
-
13 Feb 2025 1 repository listed
-
6 Feb 2025 1 repository listed
-
6 Jan 2025 1 repository listed
-
18 Dec 2024 1 repository listed
-
10 Oct 2024 1 repository listed
-
8 Oct 2024 1 repository listed
-
8 Oct 2024 1 repository listed
-
1 Oct 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 5 samples that ran constructed an object rather than computing a result (of 7 harvested samples) · 7 pointer-only (licence)
-
1 Oct 2024 1 repository listed
-
25 Sep 2024 1 repository listed
-
5 Jul 2024 1 repository listed
-
2 Jul 2024 1 repository listed
-
27 Jun 2024 1 repository listed
-
25 Jun 2024 1 repository listed
-
19 Jun 2024 1 repository listed
-
7 Jun 2024 1 repository listed
-
25 Apr 2024 1 repository listed
-
21 Apr 2024 1 repository listed Syntology official (archive's flag): 17 ran · 17 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 17 where Syntology's instrument failed) · 2 unverified (of 19 harvested samples) · 19 pointer-only (licence)
-
19 Apr 2024 1 repository listed
-
18 Mar 2024 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
23 Feb 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples) · 10 pointer-only (licence)
-
13 Feb 2024 1 repository listed
Syntology lines on 39 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections