Browse State-of-the-Art › Formal Logic

Formal Logic

20 papers with code · 1 benchmark · 3 datasets archive 2025-07-28

Knowledge BaseNatural Language ProcessingReasoning

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

DatasetBest model (first row in archive order)PaperCodeSyntologyCompare
BIG-bench (1 row) Gopher-280B (few-shot, k=5) Scaling Language Models: Methods, Analysis & Insights from Training Gopher code — Compare

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Papers archive 2025-07-28

archive papers tagged: 73 · with a code link: 20 · where Syntology ran a sample: 3 (3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (3 of 73 tagged: 3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 73 of 73, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.

Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.

  • We rethink this and adopt a well-grounded set of deduction rules based on formal logic theory, which can derive any other deduction rules when combined in a multistep way.
  • 8 Dec 2021 3 repositories listed
    Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
  • 19 Nov 2024 2 repositories listed
    Large language models (LLMs) are capable of solving a wide range of tasks, yet they have struggled with reasoning.
  • 23 Nov 2021 2 repositories listed
    In this work, we ask how we can build a rule-based system that can reason with natural language input but without the manual construction of rules.
  • 5 Jun 2025 1 repository listed
    Building directly on the deficiencies identified through our benchmark, we propose a new method, MenTaL, to address this limitation.
  • 21 May 2025 1 repository listed
    Although this data synthesis is feasible only for tasks compatible with formal verification, we observe that LLM-based PRMs trained on our dataset exhibit cross-task generalization, improving verification across diverse…
  • 21 Oct 2024 1 repository listed
    Finally, we validate CoT-TL's practicality by integrating it into a QuadCopter for multi-step drone planning based on natural language instructions.
  • 18 Sep 2024 1 repository listed
    We evaluate the framework using GPT-4o and a dataset of natural language problem descriptions, achieving 98% syntactic correctness and 88% semantic correctness.
  • 28 Aug 2024 1 repository listed
    Large language models (LLMs) are increasingly explored as general-purpose reasoners, particularly in agentic contexts.
  • 15 Apr 2024 1 repository listed
    The recent proliferation of generative artificial intelligence (AI) technologies such as pre-trained large language models (LLMs) has opened up new frontiers in computational law.
  • 3 Aug 2023 1 repository listed
    We present a comprehensive study of chart visual question-answering(QA) task, to address the challenges faced in comprehending and extracting data from chart visualizations within documents.
  • Specifically, we employ an LLM to transform natural language descriptions of logic puzzles into answer set programs.
  • Translating natural language sentences to first-order logic (NL-FOL translation) is a longstanding challenge in the NLP and formal logic literature.
  • 3 Mar 2022 1 repository listed
    With the rapidly increasing amount of scientific literature, it is getting continuously more difficult for researchers in different disciplines to be updated with the recent findings in their field of study.
  • 27 Sep 2021 1 repository listed
    In this work, we propose the use of logic AI for the design of optical quantum experiments.
  • 30 Aug 2021 1 repository listed
    Using this logic we provide a detailed map of (un)decidability and complexity results for the satisfiability and containment decision problems for different SHACL fragments.
  • 23 Dec 2020 1 repository listed
    In this paper, we describe an effective procedure to determine the negated event or property in order to replace it by its inverse.
  • 21 Aug 2020 1 repository listed
    In this paper we show, first, that the entire theory of partial information decomposition can be derived from considerations of elementary parthood relationships between information contributions.
  • 10 Oct 2019 1 repository listed
    In this paper, we present a novel verification technique that leverages Rust's type system to greatly simplify the specification and verification of system software written in Rust.
  • 26 Feb 2009 1 repository listed
    We give some lectures on the work on formal logic of Jacques Herbrand, and sketch his life and his influence on automated theorem proving.
  • LTLCrit: A Temporal Logic-based LLM Critic for Safe and Efficient Embodied Agents
    4 Jul 2025 0 repositories listed
    Large language models (LLMs) have demonstrated promise in reasoning tasks and general decision-making in static environments.
  • On Immutable Memory Systems for Artificial Agents: A Blockchain-Indexed Automata-Theoretic Framework Using ECDH-Keyed Merkle Chains
    16 Jun 2025 0 repositories listed
    This paper presents a formalised architecture for synthetic agents designed to retain immutable memory, verifiable reasoning, and constrained epistemic growth.
  • Alpay Algebra III: Observer-Coupled Collapse and the Temporal Drift of Identity
    26 May 2025 0 repositories listed
    This paper introduces a formal framework for modeling observer-dependent collapse dynamics and temporal identity drift within artificial and mathematical systems, grounded entirely in the symbolic foundations of Alpay…
  • Reasoning Under Threat: Symbolic and Neural Techniques for Cybersecurity Verification
    27 Mar 2025 0 repositories listed
    Cybersecurity demands rigorous and scalable techniques to ensure system correctness, robustness, and resilience against evolving threats.
  • Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach
    13 Mar 2025 0 repositories listed
    With the rapid advancement of Artificial Intelligence (AI), Large Language Models (LLMs) have significantly impacted a wide array of domains, including healthcare, engineering, science, education, and mathematical…
  • Building Knowledge Graphs Towards a Global Food Systems Datahub
    26 Feb 2025 0 repositories listed
    Sustainable agricultural production aligns with several sustainability goals established by the United Nations (UN).
  • Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs
    18 Feb 2025 0 repositories listed
    Linguistic evaluations of how well LMs generalize to produce or understand novel text often implicitly take for granted that natural languages are generated by symbolic rules.
  • Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Survey
    13 Feb 2025 0 repositories listed
    This comprehensive survey examines the integration of knowledge-based approaches into autonomous driving systems, with a focus on trajectory prediction and planning.
  • Automating Mathematical Proof Generation Using Large Language Model Agents and Knowledge Graphs
    4 Feb 2025 0 repositories listed
    Large Language Models have demonstrated remarkable capabilities in natural language processing tasks, including mathematical problem-solving that requires multi-step logical reasoning.
  • From An LLM Swarm To A PDDL-Empowered HIVE: Planning Self-Executed Instructions In A Multi-Modal Jungle
    17 Dec 2024 0 repositories listed
    In response to the call for agent-based solutions that leverage the ever-increasing capabilities of the deep models' ecosystem, we introduce Hive -- a comprehensive solution for selecting appropriate models and…
  • Formal Logic-guided Robust Federated Learning against Poisoning Attacks
    5 Nov 2024 0 repositories listed
  • Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence
    22 Oct 2024 0 repositories listed
  • Rulebreakers Challenge: Revealing a Blind Spot in Large Language Models' Reasoning with Formal Logic
    21 Oct 2024 0 repositories listed
  • Boosting Deductive Reasoning with Step Signals In RLHF
    12 Oct 2024 0 repositories listed
  • Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoning
    25 Sep 2024 0 repositories listed
  • ProSLM : A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering
    17 Sep 2024 0 repositories listed
  • Automated Theorem Provers Help Improve Large Language Model Reasoning
    7 Aug 2024 0 repositories listed
  • Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design
    23 Jul 2024 0 repositories listed
  • From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AI
    4 Jul 2024 0 repositories listed
  • Discussion Graph Semantics of First-Order Logic with Equality for Reasoning about Discussion and Argumentation
    18 Jun 2024 0 repositories listed
  • Procedural Adherence and Interpretability Through Neuro-Symbolic Generative Agents
    24 Feb 2024 0 repositories listed
  • Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
    22 Feb 2024 0 repositories listed
  • Inference of Abstraction for a Unified Account of Reasoning and Learning
    14 Feb 2024 0 repositories listed
  • Inference of Abstraction for a Unified Account of Symbolic Reasoning from Data
    13 Feb 2024 0 repositories listed
  • SocraSynth: Multi-LLM Reasoning with Conditional Statistics
    19 Jan 2024 0 repositories listed
  • Formal Logic Enabled Personalized Federated Learning Through Property Inference
    15 Jan 2024 0 repositories listed
  • A Simple Generative Model of Logical Reasoning and Statistical Learning
    18 May 2023 0 repositories listed
  • Generative Logic with Time: Beyond Logical Consistency and Statistical Possibility
    20 Jan 2023 0 repositories listed
  • Proceedings of the 2nd Workshop on Logic and Practice of Programming (LPOP)
    17 Nov 2022 0 repositories listed
  • Differentiable Logics for Neural Network Training and Verification
    14 Jul 2022 0 repositories listed
  • Interactive Learning from Natural Language and Demonstrations using Signal Temporal Logic
    1 Jul 2022 0 repositories listed
  • Logical Story Representations via FrameNet + Semantic Parsing
    1 Jul 2022 0 repositories listed
  • Stateless and Rule-Based Verification For Compliance Checking Applications
    14 Apr 2022 0 repositories listed
  • Towards Unifying Logical Entailment and Statistical Estimation
    27 Feb 2022 0 repositories listed
  • Logical Story Representations via FrameNet + Semantic Parsing
    16 Jan 2022 0 repositories listed
  • Learning Spatio-Temporal Specifications for Dynamical Systems
    20 Dec 2021 0 repositories listed
  • An Enactivist account of Mind Reading in Natural Language Understanding
    11 Nov 2021 0 repositories listed
  • Visual Question Answering based on Formal Logic
    8 Nov 2021 0 repositories listed
  • Expressing High-Level Scientific Claims with Formal Semantics
    27 Sep 2021 0 repositories listed
  • LGPSolver - Solving Logic Grid Puzzles Automatically
    1 Nov 2020 0 repositories listed
  • A Theory of Hypergames on Graphs for Synthesizing Dynamic Cyber Defense with Deception
    7 Aug 2020 0 repositories listed
  • Inferences and Modal Vocabulary
    6 Jul 2020 0 repositories listed
  • Machine Common Sense
    15 Jun 2020 0 repositories listed
  • Simple Dataset for Proof Method Recommendation in Isabelle/HOL (Dataset Description)
    21 Apr 2020 0 repositories listed
  • Toward the Engineering of Virtuous Machines
    7 Dec 2018 0 repositories listed
  • Consistent CCG Parsing over Multiple Sentences for Improved Logical Reasoning
    19 Apr 2018 0 repositories listed
  • Combining Representation Learning with Logic for Language Processing
    27 Dec 2017 0 repositories listed
  • Building Multimodal Simulations for Natural Language
    1 Apr 2017 0 repositories listed
  • Principles and Examples of Plausible Reasoning and Propositional Plausible Logic
    6 Mar 2017 0 repositories listed
  • The Algorithm of Islamic Jurisprudence (Fiqh) with Validation of an Entscheidungsproblem
    10 Mar 2016 0 repositories listed
  • Service Choreography, SBVR, and Time
    24 Dec 2015 0 repositories listed
  • Implementation of an Automatic Sign Language Lexical Annotation Framework based on Propositional Dynamic Logic
    25 Mar 2014 0 repositories listed
  • DLOLIS-A: Description Logic based Text Ontology Learning
    24 Mar 2013 0 repositories listed

Syntology lines on 3 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections