Browse State-of-the-Art › Known Unknowns
Known Unknowns
9 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
Language models have a tendency to generate text containing false statements that are often referred to as "Hallucinations." The primary purpose of this task is to test for this failure case by probing whether a model can correctly identify that the answer to a question is unknown. A common failure mode would be to prefer a prediction of false on unknown truth over a prediction that the answer is unknown.
Source: BIG-bench
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 15 · with a code link: 9 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 15 tagged: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 15 of 15, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from.
-
5 Apr 2022 7 repositories listed Syntology 32 ran (of which 16 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified (of 37 harvested samples)To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
24 Mar 2020 3 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)A motivating example is intensive care unit patients: the dynamics of vital physiological functions, such as the cardiovascular system with its associated variables (heart rate, cardiac contractility and output and…
-
29 Mar 2022 2 repositories listed Syntology 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 4 pointer-only (licence)We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget.
-
9 Feb 2025 1 repository listedDiscovery of high-performance materials and molecules requires identifying extremes with property values that fall outside the known distribution.
-
26 Jan 2024 1 repository listedForecasts play a central role in decision making under uncertainty.
-
23 May 2023 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)This paper investigates the capabilities of Large Language Models (LLMs) in the context of understanding their knowledge and uncertainty over questions.
-
24 Jul 2018 1 repository listedIn Experiment 1, we manipulated the presence or absence of occlusions in a director-matcher task and found that speakers spontaneously produced more informative descriptions to account for "known unknowns" in their…
-
5 Dec 2016 1 repository listedWe compare the following candidate neural network models: Maximum Likelihood, Bayesian Dropout, OSBA, and --- for MNIST --- the standard variational approximation.
-
27 Feb 2024 0 repositories listedExisting question answering (QA) datasets are no longer challenging to most powerful Large Language Models (LLMs).
-
The known unknowns of the Hsp90 chaperone31 Aug 2023 0 repositories listedMolecular chaperones are vital proteins that maintain protein homeostasis by assisting in protein folding, activation, degradation, and stress protection.
-
Machine learning for advancing low-temperature plasma modeling and simulation30 Jun 2023 0 repositories listedMachine learning has had an enormous impact in many scientific disciplines.
-
Domain Concretization from Examples: Addressing Missing Domain Knowledge via Robust Planning18 Nov 2020 0 repositories listedThe assumption of complete domain knowledge is not warranted for robot planning and decision-making in the real world.
-
Classification Uncertainty of Deep Neural Networks Based on Gradient Information22 May 2018 0 repositories listedWe study the quantification of uncertainty of Convolutional Neural Networks (CNNs) based on gradient metrics.
-
Toward Open-Set Face Recognition3 May 2017 0 repositories listedMuch research has been conducted on both face identification and face verification, with greater focus on the latter.
Syntology lines on 4 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections