Browse State-of-the-Art › LAMBADA
LAMBADA
15 papers with code · 1 benchmark · 1 dataset archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| BIG-bench (2 rows) | Chinchilla-70B (zero-shot) | Training Compute-Optimal Large Language Models | code | Syntology 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
1 dataset whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 30 · with a code link: 15 · where Syntology ran a sample: 5 (5 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (5 of 30 tagged: 5 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 30 of 30, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from.
-
17 Sep 2019 10 repositories listed Syntology community repositories only · 12 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 35 unverified (of 47 harvested samples) · 15 pointer-only (licence)To demonstrate that large language models can further advance the state of the art (SOTA), we train an 8.
-
10 Jul 2018 8 repositories listed Syntology community repositories only · 17 ran (of which 8 constructed an object rather than computing a result; 17 with no instrument failure: 1 honoured, 0 violated, 16 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified (of 25 harvested samples) · 24 pointer-only (licence)Feed-forward and convolutional architectures have recently been shown to achieve superior results on some sequence modeling tasks such as machine translation, with the added advantage that they concurrently process all…
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
20 Jun 2016 3 repositories listedWe introduce LAMBADA, a dataset to evaluate the capabilities of computational models for text understanding by means of a word prediction task.
-
29 Mar 2022 2 repositories listed Syntology 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 4 pointer-only (licence)We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget.
-
6 Apr 2020 2 repositories listedAttention is a commonly used mechanism in sequence processing, but it is of O(n^2) complexity which prevents its application to long sequences.
-
3 Nov 2024 1 repository listedIn FactualityPrompts, an open-ended text generation benchmark, sampling using APD significantly boosts factuality in comparison to the CD sampling and its variants, and achieves state-of-the-art results for Pythia 6.
-
28 Oct 2024 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 4 honoured, 0 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 10 harvested samples) · 10 pointer-only (licence)Moreover, we demonstrate the efficacy of our approach for diffusion language models with up to 860M parameters.
-
17 Jul 2024 1 repository listedRapid advancements in GPU computational power has outpaced memory capacity and bandwidth growth, creating bottlenecks in Large Language Model (LLM) inference.
-
30 Dec 2022 1 repository listedLearning to predict masked tokens in a sequence has been shown to be a helpful pretraining objective for powerful language models such as PaLM2.
-
13 Aug 2021 1 repository listedTo reduce the wall-clock training time, a common practice is to increase the batch size and learning rate.
-
1 Dec 2019 1 repository listedA key requirement in sequence to sequence processing is the modeling of long range dependencies.
-
8 Nov 2019 1 repository listedBased on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks.
-
18 Jul 2019 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)A key requirement in sequence to sequence processing is the modeling of long range dependencies.
-
5 Oct 2018 1 repository listedReading comprehension tasks test the ability of models to process long-term context and remember salient information.
-
Matryoshka Model Learning for Improved Elastic Student Models29 May 2025 0 repositories listedIndustry-grade ML models are carefully designed to meet rapidly evolving serving constraints, which requires significant resources for model development.
-
AdaGC: Improving Training Stability for Large Language Model Pretraining16 Feb 2025 0 repositories listedLarge Language Models (LLMs) face increasing loss spikes during scaling, undermining training stability and final performance.
-
SymBa: Symbolic Backward Chaining for Structured Natural Language Reasoning20 Feb 2024 0 repositories listedTo improve the performance and explainability of LLM-based natural language reasoning, structured reasoning can be applied to generate explicitly structured proofs.
-
PIXAR: Auto-Regressive Language Modeling in Pixel Space6 Jan 2024 0 repositories listedRecent work showed the possibility of building open-vocabulary large language models (LLMs) that directly operate on pixel representations.
-
Concise and Organized Perception Facilitates Reasoning in Large Language Models5 Oct 2023 0 repositories listedExploiting large language models (LLMs) to tackle reasoning has garnered growing attention.
-
Headless Language Models: Learning without Predicting with Contrastive Weight Tying15 Sep 2023 0 repositories listedSelf-supervised pre-training of language models usually consists in predicting probability distributions over extensive token vocabularies.
-
30 Jun 2023 0 repositories listedClassifier-Free Guidance (CFG) has recently emerged in text-to-image generation as a lightweight technique to encourage prompt-adherence in generations.
-
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language20 Dec 2022 0 repositories listedRemarkable progress has been made on automated reasoning with natural text, by using Language Models (LMs) and methods such as Chain-of-Thought and Selection-Inference.
-
Leveraging Relaxed Equilibrium by Lazy Transition for Sequence Modeling1 May 2022 0 repositories listedIn sequence modeling, certain tokens are usually less ambiguous than others, and representations of these tokens require fewer refinements for disambiguation.
-
CoreLM: Coreference-aware Language Model Fine-Tuning4 Nov 2021 0 repositories listedLanguage Models are the underpin of all modern Natural Language Processing (NLP) tasks.
-
E.T.: Entity-Transformers. Coreference augmented Neural Language Model for richer mention representations via Entity-Transformer blocks10 Nov 2020 0 repositories listedIn the last decade, the field of Neural Language Modelling has witnessed enormous changes, with the development of novel models through the use of Transformer architectures.
-
Attending to Entities for Better Text Understanding11 Nov 2019 0 repositories listedRecent progress in NLP witnessed the development of large-scale pre-trained language models (GPT, BERT, XLNet, etc.)
-
16 Apr 2018 0 repositories listedMany problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text.
-
7 Mar 2017 0 repositories listedTraining recurrent neural networks to model long term dependencies is difficult.
-
26 Oct 2016 0 repositories listedProgress in text understanding has been driven by large datasets that test particular capabilities, like recent datasets for reading comprehension (Hermann et al., 2015).
Syntology lines on 5 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections