WinoGrande
Introduced by Keisuke Sakaguchi et al. in
WinoGrande: An Adversarial Winograd Schema Challenge at Scale archive 2025-07-28
WinoGrande is a large-scale dataset of 44k problems, inspired by the original WSC design, but adjusted to improve both the scale and the hardness of the dataset. The key steps of the dataset construction consist of (1) a carefully designed crowdsourcing procedure, followed by (2) systematic bias reduction using a novel AfLite algorithm that generalizes human-detectable word associations to machine-detectable embedding associations.
Source: WinoGrande: An Adversarial Winograd Schema Challenge at Scale
Image Source: https://winogrande.allenai.org/
Benchmarks archive 2025-07-28
All 7 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
Syntology 33 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 1 placed): Common Sense Reasoning · WinoGrande (31 judged, 1 placed), parameter-efficient fine-tuning · WinoGrande (8 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Papers archive 2025-07-28
papers with a benchmark row: 34 · with a code link: 30 · where Syntology ran a sample: 17 (14 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show:
all papers with a benchmark row only where code ran (17 of 34 with a benchmark row: 14 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 34 of 34 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).
The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 703. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed 2026-10-05 ; when this build read Syntology's graph is in the build record .
Paper Code Results Date Samples run Syntology
GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs
1
1
27 Aug 2024
official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (5 pointer-only for licence )
Mixture-of-Subspaces in Low-Rank Adaptation
1
1
16 Jun 2024
official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (6 pointer-only for licence )
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
2
3
22 Apr 2024
official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
1
1
12 Mar 2024
not harvested
The Claude 3 Model Family: Opus, Sonnet, Haiku
0
3
4 Mar 2024
not harvested
DoRA: Weight-Decomposed Low-Rank Adaptation
5
1
14 Feb 2024
official (archive's flag): 7 ran · 8 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 8 where Syntology's instrument failed) · 7 unverified (14 pointer-only for licence )
Mixtral of Experts
6
2
8 Jan 2024
5 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 5 samples that ran constructed an object rather than computing a result
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
2
1
5 Jan 2024
not harvested
Mistral 7B
6
1
10 Oct 2023
official (archive's flag): 9 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence )
Textbooks Are All You Need II: phi-1.5 technical report
1
1
11 Sep 2023
not harvested
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning
2
1
23 May 2023
not harvested
PaLM 2 Technical Report
1
3
17 May 2023
not harvested
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
1
6
27 Apr 2023
not harvested
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
4
4
3 Apr 2023
not harvested
BloombergGPT: A Large Language Model for Finance
2
4
30 Mar 2023
not harvested
GPT-4 Technical Report
11
2
15 Mar 2023
community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence )
LLaMA: Open and Efficient Foundation Language Models
57
4
27 Feb 2023
official: no sample here; runs from other or unrecorded repositories · 37 ran (of which 9 constructed an object rather than computing a result; 25 with no instrument failure: 3 honoured, 0 violated, 22 with no contract checked; 12 where Syntology's instrument failed) · 21 unverified (4 pointer-only for licence )
Exploring the Benefits of Training Expert Language Models over Instruction Tuning
2
1
7 Feb 2023
official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence )
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
0
1
28 Oct 2022
not harvested
Task Compass: Scaling Multi-task Pre-training with Task Prefix
1
3
12 Oct 2022
not harvested
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners
1
1
6 Oct 2022
not harvested
PaLM: Scaling Language Modeling with Pathways
7
3
5 Apr 2022
32 ran (of which 16 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified
Training Compute-Optimal Large Language Models
2
1
29 Mar 2022
8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (4 pointer-only for licence )
Efficient Language Modeling with Sparse all-MLP
0
5
14 Mar 2022
not harvested
ST-MoE: Designing Stable and Transferable Sparse Expert Models
3
2
17 Feb 2022
community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (5 pointer-only for licence )
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
3
1
8 Dec 2021
not harvested
Finetuned Language Models Are Zero-Shot Learners
8
2
3 Sep 2021
official: harvested for another paper · 0 ran · 1 unverified
LoRA: Low-Rank Adaptation of Large Language Models
74
1
17 Jun 2021
community repositories only · 51 ran (of which 19 constructed an object rather than computing a result; 43 with no instrument failure: 1 honoured, 0 violated, 42 with no contract checked; 8 where Syntology's instrument failed) · 33 unverified (30 pointer-only for licence )
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema
0
7
16 Apr 2021
not harvested
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark
1
1
24 Mar 2021
official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
Language Models are Few-Shot Learners
67
2
28 May 2020
community repositories only · 45 ran (of which 0 constructed an object rather than computing a result; 37 with no instrument failure: 3 honoured, 2 violated, 32 with no contract checked; 8 where Syntology's instrument failed) · 20 unverified (7 pointer-only for licence )
UnifiedQA: Crossing Format Boundaries With a Single QA System
2
2
2 May 2020
official (archive's flag): 4 ran · 4 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (1 pointer-only for licence )
Generative Data Augmentation for Commonsense Reasoning
1
1
24 Apr 2020
official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (4 pointer-only for licence )
WinoGrande: An Adversarial Winograd Schema Challenge at Scale
10
6
24 Jul 2019
not harvested
Dataset loaders archive 2025-07-28
6 loaders as listed in the archive; links are outbound and not re-checked here.
Tasks archive 2025-07-28
License archive 2025-07-28
CC-BY
Modalities archive 2025-07-28
Languages archive 2025-07-28
No language tagged.
Variants archive 2025-07-28
WinoGrande Winogrande Winogrande (5-shot) Winogrande TR v0.2 Winogrande TR
5 variant names, as the archive lists them.