Browse State-of-the-Art › Formal Logic
Formal Logic
20 papers with code · 1 benchmark · 3 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| BIG-bench (1 row) | Gopher-280B (few-shot, k=5) | Scaling Language Models: Methods, Analysis & Insights from Training Gopher | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
3 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 73 · with a code link: 20 · where Syntology ran a sample: 3 (3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (3 of 73 tagged: 3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 73 of 73, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
11 Aug 2023 3 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples)We rethink this and adopt a well-grounded set of deduction rules based on formal logic theory, which can derive any other deduction rules when combined in a multistep way.
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
19 Nov 2024 2 repositories listedLarge language models (LLMs) are capable of solving a wide range of tasks, yet they have struggled with reasoning.
-
23 Nov 2021 2 repositories listedIn this work, we ask how we can build a rule-based system that can reason with natural language input but without the manual construction of rules.
-
5 Jun 2025 1 repository listedBuilding directly on the deficiencies identified through our benchmark, we propose a new method, MenTaL, to address this limitation.
-
21 May 2025 1 repository listedAlthough this data synthesis is feasible only for tasks compatible with formal verification, we observe that LLM-based PRMs trained on our dataset exhibit cross-task generalization, improving verification across diverse…
-
21 Oct 2024 1 repository listedFinally, we validate CoT-TL's practicality by integrating it into a QuadCopter for multi-step drone planning based on natural language instructions.
-
18 Sep 2024 1 repository listedWe evaluate the framework using GPT-4o and a dataset of natural language problem descriptions, achieving 98% syntactic correctness and 88% semantic correctness.
-
28 Aug 2024 1 repository listedLarge language models (LLMs) are increasingly explored as general-purpose reasoners, particularly in agentic contexts.
-
15 Apr 2024 1 repository listedThe recent proliferation of generative artificial intelligence (AI) technologies such as pre-trained large language models (LLMs) has opened up new frontiers in computational law.
-
3 Aug 2023 1 repository listedWe present a comprehensive study of chart visual question-answering(QA) task, to address the challenges faced in comprehending and extracting data from chart visualizations within documents.
-
15 Jul 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)Specifically, we employ an LLM to transform natural language descriptions of logic puzzles into answer set programs.
-
24 May 2023 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)Translating natural language sentences to first-order logic (NL-FOL translation) is a longstanding challenge in the NLP and formal logic literature.
-
3 Mar 2022 1 repository listedWith the rapidly increasing amount of scientific literature, it is getting continuously more difficult for researchers in different disciplines to be updated with the recent findings in their field of study.
-
27 Sep 2021 1 repository listedIn this work, we propose the use of logic AI for the design of optical quantum experiments.
-
30 Aug 2021 1 repository listedUsing this logic we provide a detailed map of (un)decidability and complexity results for the satisfiability and containment decision problems for different SHACL fragments.
-
23 Dec 2020 1 repository listedIn this paper, we describe an effective procedure to determine the negated event or property in order to replace it by its inverse.
-
21 Aug 2020 1 repository listedIn this paper we show, first, that the entire theory of partial information decomposition can be derived from considerations of elementary parthood relationships between information contributions.
-
10 Oct 2019 1 repository listedIn this paper, we present a novel verification technique that leverages Rust's type system to greatly simplify the specification and verification of system software written in Rust.
-
26 Feb 2009 1 repository listedWe give some lectures on the work on formal logic of Jacques Herbrand, and sketch his life and his influence on automated theorem proving.
-
LTLCrit: A Temporal Logic-based LLM Critic for Safe and Efficient Embodied Agents4 Jul 2025 0 repositories listedLarge language models (LLMs) have demonstrated promise in reasoning tasks and general decision-making in static environments.
-
On Immutable Memory Systems for Artificial Agents: A Blockchain-Indexed Automata-Theoretic Framework Using ECDH-Keyed Merkle Chains16 Jun 2025 0 repositories listedThis paper presents a formalised architecture for synthetic agents designed to retain immutable memory, verifiable reasoning, and constrained epistemic growth.
-
Alpay Algebra III: Observer-Coupled Collapse and the Temporal Drift of Identity26 May 2025 0 repositories listedThis paper introduces a formal framework for modeling observer-dependent collapse dynamics and temporal identity drift within artificial and mathematical systems, grounded entirely in the symbolic foundations of Alpay…
-
Reasoning Under Threat: Symbolic and Neural Techniques for Cybersecurity Verification27 Mar 2025 0 repositories listedCybersecurity demands rigorous and scalable techniques to ensure system correctness, robustness, and resilience against evolving threats.
-
Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach13 Mar 2025 0 repositories listedWith the rapid advancement of Artificial Intelligence (AI), Large Language Models (LLMs) have significantly impacted a wide array of domains, including healthcare, engineering, science, education, and mathematical…
-
Building Knowledge Graphs Towards a Global Food Systems Datahub26 Feb 2025 0 repositories listedSustainable agricultural production aligns with several sustainability goals established by the United Nations (UN).
-
Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs18 Feb 2025 0 repositories listedLinguistic evaluations of how well LMs generalize to produce or understand novel text often implicitly take for granted that natural languages are generated by symbolic rules.
-
Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Survey13 Feb 2025 0 repositories listedThis comprehensive survey examines the integration of knowledge-based approaches into autonomous driving systems, with a focus on trajectory prediction and planning.
-
Automating Mathematical Proof Generation Using Large Language Model Agents and Knowledge Graphs4 Feb 2025 0 repositories listedLarge Language Models have demonstrated remarkable capabilities in natural language processing tasks, including mathematical problem-solving that requires multi-step logical reasoning.
-
From An LLM Swarm To A PDDL-Empowered HIVE: Planning Self-Executed Instructions In A Multi-Modal Jungle17 Dec 2024 0 repositories listedIn response to the call for agent-based solutions that leverage the ever-increasing capabilities of the deep models' ecosystem, we introduce Hive -- a comprehensive solution for selecting appropriate models and…
-
Formal Logic-guided Robust Federated Learning against Poisoning Attacks5 Nov 2024 0 repositories listed
-
Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence22 Oct 2024 0 repositories listed
-
Rulebreakers Challenge: Revealing a Blind Spot in Large Language Models' Reasoning with Formal Logic21 Oct 2024 0 repositories listed
-
Boosting Deductive Reasoning with Step Signals In RLHF12 Oct 2024 0 repositories listed
-
Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoning25 Sep 2024 0 repositories listed
-
ProSLM : A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering17 Sep 2024 0 repositories listed
-
Automated Theorem Provers Help Improve Large Language Model Reasoning7 Aug 2024 0 repositories listed
-
Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design23 Jul 2024 0 repositories listed
-
From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AI4 Jul 2024 0 repositories listed
-
Discussion Graph Semantics of First-Order Logic with Equality for Reasoning about Discussion and Argumentation18 Jun 2024 0 repositories listed
-
Procedural Adherence and Interpretability Through Neuro-Symbolic Generative Agents24 Feb 2024 0 repositories listed
-
Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic22 Feb 2024 0 repositories listed
-
Inference of Abstraction for a Unified Account of Reasoning and Learning14 Feb 2024 0 repositories listed
-
Inference of Abstraction for a Unified Account of Symbolic Reasoning from Data13 Feb 2024 0 repositories listed
-
SocraSynth: Multi-LLM Reasoning with Conditional Statistics19 Jan 2024 0 repositories listed
-
Formal Logic Enabled Personalized Federated Learning Through Property Inference15 Jan 2024 0 repositories listed
-
A Simple Generative Model of Logical Reasoning and Statistical Learning18 May 2023 0 repositories listed
-
Generative Logic with Time: Beyond Logical Consistency and Statistical Possibility20 Jan 2023 0 repositories listed
-
Proceedings of the 2nd Workshop on Logic and Practice of Programming (LPOP)17 Nov 2022 0 repositories listed
-
Differentiable Logics for Neural Network Training and Verification14 Jul 2022 0 repositories listed
-
Interactive Learning from Natural Language and Demonstrations using Signal Temporal Logic1 Jul 2022 0 repositories listed
-
Logical Story Representations via FrameNet + Semantic Parsing1 Jul 2022 0 repositories listed
-
Stateless and Rule-Based Verification For Compliance Checking Applications14 Apr 2022 0 repositories listed
-
Towards Unifying Logical Entailment and Statistical Estimation27 Feb 2022 0 repositories listed
-
Logical Story Representations via FrameNet + Semantic Parsing16 Jan 2022 0 repositories listed
-
Learning Spatio-Temporal Specifications for Dynamical Systems20 Dec 2021 0 repositories listed
-
An Enactivist account of Mind Reading in Natural Language Understanding11 Nov 2021 0 repositories listed
-
Visual Question Answering based on Formal Logic8 Nov 2021 0 repositories listed
-
Expressing High-Level Scientific Claims with Formal Semantics27 Sep 2021 0 repositories listed
-
LGPSolver - Solving Logic Grid Puzzles Automatically1 Nov 2020 0 repositories listed
-
A Theory of Hypergames on Graphs for Synthesizing Dynamic Cyber Defense with Deception7 Aug 2020 0 repositories listed
-
Inferences and Modal Vocabulary6 Jul 2020 0 repositories listed
-
Machine Common Sense15 Jun 2020 0 repositories listed
-
Simple Dataset for Proof Method Recommendation in Isabelle/HOL (Dataset Description)21 Apr 2020 0 repositories listed
-
Toward the Engineering of Virtuous Machines7 Dec 2018 0 repositories listed
-
Consistent CCG Parsing over Multiple Sentences for Improved Logical Reasoning19 Apr 2018 0 repositories listed
-
Combining Representation Learning with Logic for Language Processing27 Dec 2017 0 repositories listed
-
Building Multimodal Simulations for Natural Language1 Apr 2017 0 repositories listed
-
Principles and Examples of Plausible Reasoning and Propositional Plausible Logic6 Mar 2017 0 repositories listed
-
The Algorithm of Islamic Jurisprudence (Fiqh) with Validation of an Entscheidungsproblem10 Mar 2016 0 repositories listed
-
Service Choreography, SBVR, and Time24 Dec 2015 0 repositories listed
-
Implementation of an Automatic Sign Language Lexical Annotation Framework based on Propositional Dynamic Logic25 Mar 2014 0 repositories listed
-
DLOLIS-A: Description Logic based Text Ontology Learning24 Mar 2013 0 repositories listed
Syntology lines on 3 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections