Datasets › KILT

KILT (KILT Benchmark)

Introduced by Fabio Petroni et al. in KILT: a Benchmark for Knowledge Intensive Language Tasks4 Sep 2020 archive 2025-07-28

KILT (Knowledge Intensive Language Tasks) is a benchmark consisting of 11 datasets representing 5 types of tasks:

  • Fact-checking (FEVER),
  • Entity linking (AIDA CoNLL-YAGO, WNED-WIKI, WNED-CWEB),
  • Slot filling (T-Rex, Zero Shot RE),
  • Open domain QA (Natural Questions, HotpotQA, TriviaQA, ELI5),
  • Dialog generation (Wizard of Wikipedia).

All these datasets have been grounded in a single pre-processed wikipedia snapshot, allowing for fairer and more consistent evaluation as well as enabling new task setups such as multitask and transfer learning.

Source: KILT Benchmarking

Benchmarks archive 2025-07-28

All 11 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

First row (archive order)PaperCode
Fact Verification KILT: FEVER Re2G KILT-AC 78.53 Re2G: Retrieve, Rerank, Generate ibm/kgi-slot-filling 33 Compare
Open-Domain Dialog KILT: Wizard of Wikipedia Hindsight KILT-RL 11.92 — — 21 Compare
Slot Filling KILT: Zero Shot RE single ngram KILT-AC 73.2 — — 21 Compare
Slot Filling KILT: T-REx Re2G KILT-AC 75.84 Re2G: Retrieve, Rerank, Generate ibm/kgi-slot-filling 20 Compare
Open-Domain Question Answering KILT: Natural Questions Re2G KILT-EM 43.56 Re2G: Retrieve, Rerank, Generate ibm/kgi-slot-filling 16 Compare
Open-Domain Question Answering KILT: ELI5 somebody KILT-RL 2.62 — — 16 Compare
Open-Domain Question Answering KILT: HotpotQA intersect KILT-EM 18.06 — — 14 Compare
Entity Linking KILT: AIDA-YAGO2 GENRE KILT-AC 89.85 Autoregressive Entity Retrieval facebookresearch/GENRE +1 11 Compare
Entity Linking KILT: WNED-CWEB GENRE KILT-AC 71.22 Autoregressive Entity Retrieval facebookresearch/GENRE +1 10 Compare
Entity Linking KILT: WNED-WIKI GENRE KILT-AC 87.44 Autoregressive Entity Retrieval facebookresearch/GENRE +1 10 Compare
Question Answering KILT: ELI5 RBG Rouge-L 27.13 Read before Generate! Faithful Long Form Question... — 7 Compare

Papers archive 2025-07-28

papers with a benchmark row: 8 · with a code link: 7 · where Syntology ran a sample: 3 (3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (3 of 8 with a benchmark row: 3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 8 of 8 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 117. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

Dataset loaders archive 2025-07-28

2 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

License archive 2025-07-28

No licence recorded in the archive. Absence here is not a statement about the dataset's terms.

Modalities archive 2025-07-28

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • NQ KILT
  • KILT: Zero Shot RE
  • KILT: Wizard of Wikipedia
  • KILT: WNED-WIKI
  • KILT: WNED-CWEB
  • KILT: TriviaQA
  • KILT: T-REx
  • KILT: Natural Questions
  • KILT: HotpotQA
  • KILT: FEVER
  • KILT: ELI5
  • KILT: AIDA-YAGO2
  • KILT

13 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections