What is not shown
A site that only shows what it has cannot be checked. This page is the build's exclusion ledger: what the archive and the graph hold that no page renders, with the number and the reason. Numbers are from this build; the archive snapshot is 2025-07-28. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.
The same ledger as JSON, built from the dict this page prints: /not-shown.json.
Pages
| What | Count | Why |
|---|---|---|
| papers in the archive catalogue | 625,187 | abstracts table plus links-only rows |
| paper pages rendered | 225,441 | papers with a code link, a benchmark row, an introduced dataset or method, or Syntology samples |
| papers without a page | 399,746 | abstract-only: nothing runnable or ranked to show; the search index does not list them |
| method entries removed as spam (name pattern) | 12,604 | customer-service phrasing in the name, counted over the methods table and the paper catalogue's embedded method list; the pattern is public in the build script |
| duplicate leaderboard tables collapsed | 10,354 | the same benchmark reached from several task nodes |
| method entries removed as spam (dummy-paper clusters) | 6,785 | entries hung off five dummy paper records, counted over the methods table and the paper catalogue's embedded method list; see the attribution page |
| dataset rows removed as spam | 2,794 | contact-centre spam in the dataset's name or full name; the pattern is public in the build script |
| papers excluded for having no title | 636 | a page needs a title |
| per-paper method tags dropped as spam | 37,766 | the method-name pattern, applied to the tags the paper catalogue carries |
| dataset descriptions blanked as vandalised | 201 | row kept, text removed |
| method-collection descriptions blanked as vandalised | 65 | text removed |
| method descriptions blanked as vandalised | 22 | row kept, text removed |
| task descriptions blanked as vandalised | 5 | row kept, text removed |
| per-paper method tags with no method page | 4,230 | shown as plain text, not linked |
| per-paper task tags with no task page | 29,815 | not shown; the archive tagged papers with labels that never had a task page |
| benchmark dataset nodes with no table | 190 | no metrics and no rows |
| leaderboard rows with no paper page | 3,740 | row has no paper URL, or a URL that matches no catalogue paper |
| benchmarks whose dataset has no dataset page | 3,096 | name matched nothing in the dataset catalogue |
| task slugs derived rather than archived | 1,324 | the archive's task URLs cover only tasks that a dataset lists |
| Syntology: paper pages whose arXiv id is not in the Atlas id list | 108,712 | checked against the Atlas id list (549,449 ids, as this build read them; the read time is in the build record); these pages link to no Syntology page and do not say the graph lacks the paper, because the list is not the whole graph (7,185 papers in this site's copy of the graph have an arXiv id the list lacks); Syntology's feed shows papers as they enter the graph |
| Syntology: paper pages that cannot be joined (no arXiv id) | 12,898 | this site joins papers to Syntology's graph by arXiv id; these pages do not say whether the graph holds the paper |
| papers in Syntology's layer with harvested samples and no page here Syntology | 2,441 | Of these, 1,954 carry an id in the export that is not an arXiv id (731 ACL Anthology ids, 489 CVF open access ids, 439 AAAI ids, 117 IJCAI ids, 96 OpenReview ids, 82 bare-number ids), and 487 have an arXiv id from before the snapshot that the archive catalogue does not hold. Their 18,061 sample links, and the 2,343 of these papers with a sample that ran, are counted in the site's sample and paper totals. None of them is given a page or a link into Syntology from here; their ids are repaired where the graph layer is exported, not on this site. |
| ACL Anthology ids (not arXiv ids) | 731 | the graph layer's arxiv_id field holds an id of this kind, not an arXiv id |
| CVF open access ids (not arXiv ids) | 489 | the graph layer's arxiv_id field holds an id of this kind, not an arXiv id |
| AAAI ids (not arXiv ids) | 439 | the graph layer's arxiv_id field holds an id of this kind, not an arXiv id |
| IJCAI ids (not arXiv ids) | 117 | the graph layer's arxiv_id field holds an id of this kind, not an arXiv id |
| OpenReview ids (not arXiv ids) | 96 | the graph layer's arxiv_id field holds an id of this kind, not an arXiv id |
| bare-number ids (not arXiv ids) | 82 | the graph layer's arxiv_id field holds an id of this kind, not an arXiv id |
| arXiv id from before the snapshot, not in the archive catalogue | 487 | neither an archive page nor a page for a paper newer than the archive |
Withheld or changed by this site's content rules
Archive rows and fields that the site's own content rules withheld, blanked or changed at build time, each count the rules keep, listed once. Rules: cwp/cwp_lib/data.py PROSE_RULES / NAME_RULES (NFKC-folded text); applied at build time to TEMP copies of the catalog tables, never to the catalog file.
| What | Count |
|---|---|
| dataset rows withheld, all reasons | 40 |
| dataset rows withheld: affiliate or product advertising | 24 |
| dataset rows withheld: adult | 4 |
| dataset rows withheld: travel-booking or airline-support spam | 4 |
| dataset rows withheld: gambling advertising | 4 |
| dataset rows withheld: payment or crypto support | 2 |
| dataset rows withheld: tech support | 1 |
| dataset rows withheld: contact-centre spam | 1 |
| method entries withheld, all reasons | 32 |
| method entries withheld: contact-centre spam | 10 |
| method entries withheld: travel-booking or airline-support spam | 7 |
| method entries withheld: affiliate or product advertising | 6 |
| method entries withheld: gambling advertising | 3 |
| method entries withheld: pharmacy advertising | 2 |
| method entries withheld: a test entry | 2 |
| method entries withheld: essay mill | 1 |
| method entries withheld: name and text replaced by an advert (affiliate or product advertising) | 1 |
| task entries withheld, all reasons | 2 |
| task entries withheld: contact-centre spam | 2 |
| leaderboard tables withheld, all reasons | 27 |
| leaderboard tables withheld: its only row was a test entry | 23 |
| leaderboard tables withheld: every row was withheld as spam | 3 |
| leaderboard tables withheld: its task was withheld | 1 |
| leaderboard rows withheld, all reasons | 26 |
| leaderboard rows withheld: a single row credited to a dummy paper record | 23 |
| leaderboard rows withheld: contact-centre spam | 2 |
| leaderboard rows withheld: travel-booking or airline-support spam | 1 |
| dataset rows: description blanked, row kept: llm paste | 35 |
| method entries: description blanked, row kept: one junk word as the whole text | 2 |
| dataset rows: licence link blanked, row kept: it pointed at a spam or darknet-market host | 1 |
| dataset rows: description blanked, row kept: darknet advertising | 1 |
| dataset rows: licence name blanked, row kept: it repeated the spam name that replaced the dataset's own name in the same edit (the name was restored from the dataset's address; the licence link was removed with it) | 1 |
| method entries: description blanked, row kept: a phone number | 1 |
| task entries: description blanked, row kept: a phone number | 1 |
| dataset names changed in the same edit as a spam description, restored from the dataset's address | 2 |
| distinct method tag names on papers withheld as spam | 255 |
| distinct task tag names on papers withheld as spam | 1 |
| method "see code" links shown as plain text: not a code host | 26 |
| method source links shown as plain text: not a paper host | 12 |
What the archive cannot supply at all
- GitHub stars, "trending", and library pages: never in the published tables.
- Thumbnails and dataset images: hosted on the original site's media server, which no longer resolves.
- Whether a metric is higher-better or lower-better: not recorded in the archive. Leaderboards keep the archive's row order as rank; the trend chart infers a direction from the metric's name and says so, or draws points only.
- Any leaderboard row added after 2025-07-28. Results Syntology extracted from newer papers are listed separately below a table's archive rows (next section), never as archive rows.
Results Syntology placed on archive leaderboards Syntology
Syntology has checked 8,886 of the 18,637 papers on this site that are newer than the archive (for 3,717 of them no archive leaderboard matched the paper's tables, so there was nothing further to check); 733 were read and have nothing to place (arXiv has no HTML version of the paper, or that version has no tables), 41 could not be read (the extractor's reply could not be parsed), and results from the other 8,977 appear after they are checked.
Every paper the leaderboard extractor read, and what became of each leaderboard it proposed. A model pointed at the cell in the paper's own table; the number was read from that cell and checked against the leaderboard's metric, dataset, split and scale; an independent check that saw the leaderboard's other rows and every other leaderboard on the same dataset accepted or rejected the entry. Only accepted entries are shown anywhere on the site.
| What | Count | Why |
|---|---|---|
| papers on this site newer than the archive | 18,637 | the extractor reads every one, and the independent check follows wherever it proposes a leaderboard; counted from this build's graph layer |
| checked | 8,886 | the extractor read the paper's tables and compared them with the archive's leaderboards, and the independent check gave a verdict (accept or reject) on every leaderboard the extractor proposed; for 3,717 of these papers no archive leaderboard matched the tables, so none was proposed; a paper checked in part can also show a result, on the leaderboards the check accepted |
| read, nothing to place | 733 | arXiv has no HTML version of the paper (260), or that version has no tables (473); there is no table to place a result from, so these are not counted as checked |
| could not be read | 41 | the extractor's reply could not be parsed; not counted as checked, and nothing from these papers is shown |
| read by the extractor, not yet checked | 0 | nothing from these papers is shown until the check has run |
| checked in part | 2 | the check gave no verdict on at least one of the paper's leaderboards; nothing unchecked is shown |
| not yet read by the extractor | 8,975 | the extractor has not reached the paper yet, or its last attempt stopped before writing a result and the paper is read again on the next run |
| leaderboards awaiting a verdict | 3 | (paper, leaderboard) pairs with rule-passing cells the independent check has not yet accepted or rejected; none is shown |
| papers with a line in the verified file | 9,662 | one line per paper the extractor finished with, whatever the outcome, read once at build time (29,998,752 bytes, sha256 e64d2b4c9755…); by outcome below |
| papers: answered | 5,171 | the extractor answered for the paper's tables |
| papers: no candidate board | 3,717 | no archive leaderboard plausibly matches the paper's tables |
| papers: no tables | 473 | the paper's HTML rendering has no tables |
| papers: no html | 260 | arXiv holds no HTML rendering of the paper, so it has no readable tables |
| papers: unparseable | 41 | the extractor's reply could not be parsed; counted as could not be read, never as checked |
| leaderboards proposed | 4,465 | (paper, leaderboard) pairs the model said report the paper's own method on that leaderboard |
| refusals by a rule | 9,670 | each cell, row or configuration the model pointed at that failed a deterministic check, counted once under the first check it failed (below); none is shown |
| rule: metric not in column header | 3,803 | the table's column header does not name the leaderboard's metric |
| rule: cell text mismatch | 1,224 | the number the model quoted is not the number printed in the cell it pointed at |
| rule: dataset not in table | 849 | the table does not name the leaderboard's dataset |
| rule: scale mismatch | 749 | the value is not on the scale of the leaderboard column's archive values |
| rule: no quoted value | 648 | the model gave no value to check against the cell |
| rule: label mismatch | 628 | the row label the model quoted is not the row's label |
| rule: ablation label | 547 | the row is an ablation variant, not the method itself |
| rule: one configuration two values | 273 | one configuration gave two values for one column |
| rule: split mismatch | 265 | the table reports a different split or setting than the leaderboard |
| rule: header row | 206 | the model pointed at a header row |
| rule: several configurations no direction | 157 | several configurations of the method, and no metric direction to pick one |
| rule: quoted value ambiguous in row | 141 | the quoted value appears more than once in the row |
| rule: metric qualifier in header | 70 | the column header qualifies the metric in a way the leaderboard does not |
| rule: board column not on board | 36 | the column named is not one of the leaderboard's columns |
| rule: bad coordinates | 34 | the table, row or column pointed at does not exist |
| rule: header names two metrics | 27 | the column header names two metrics |
| rule: not a configuration row | 13 | the row is not a configuration of the method |
| independent check: reject | 969 | rejected by the independent check |
| independent check: accept | 132 | accepted by the independent check; placed on the leaderboard |
| independent check: unparseable | 3 | the independent check's reply could not be parsed; not placed |
| entries placed on leaderboards | 132 | one per paper per leaderboard; only these render |
| cells placed | 160 | the numbers those entries print, one per leaderboard column |
| leaderboards carrying an entry | 95 | listed below the archive rows, never ranked among them |
| papers with an entry | 93 | each has a paper page that says where |
Every leaderboard carrying a placed result, with its number of entries and newest paper: leaderboards with results placed since the archive.
Measured before publishing, on held-out archive papers: the extractor was given 883 of them and produced output for 881 (the other 2 had no title and were excluded before any model call, so they are not part of this measurement). A model adjudicator (Claude Opus 5.5), blind to which entries were real, judged 108 of 110 accepted entries correct on those held-out archive papers; its own controls: 69 of 70 archive-confirmed entries judged true, 101 of 103 corrupted copies judged false. It was not measured on the newer papers whose results are placed on this site. The rules were frozen before that test was run.
Papers newer than the archive
| What | Count | Why |
|---|---|---|
| post-freeze paper pages | 18,637 | a repository link in Syntology's graph, or (0 of them) no repository but an extracted result attached to an archive table |
| with arXiv's metadata (title, abstract, authors, date) | 18,637 | arXiv metadata, CC0 1.0 |
| without arXiv's metadata | 0 | arXiv's metadata file holds no record for them yet; the page shows the graph's title and says authors are on arXiv |
Changed after a reader report
No decided report changed anything in this build.
Syntology's layer, in numbers
Two units, named on every row. A sample is one distinct piece of code, counted once however many papers' repositories hold it; these are the same counts Code prints. A link is one sample under one paper, so identical code held for several papers is one sample and several links. Licence and where a copy came from are properties of the link; status and fingerprint are properties of the sample.
| What | Count | Unit |
|---|---|---|
| papers in Syntology's layer | 63,466 | papers |
| papers with harvested code samples | 51,887 | papers |
| papers newer than the archive with a repository link in the graph | 18,637 | papers |
| distinct code samples | 294,260 | samples |
| samples that ran (any status except unverified) | 211,169 | samples |
| of which checked (honoured, violated, or ran with no contract) | 147,384 | samples |
| of which Syntology's instrument failed (our draft was wrong, or the contract check could not run): failures of Syntology's instrument, not of the code | 63,785 | samples |
| ran, honoured the contract we drafted | 15,745 | samples |
| ran, violated the contract we drafted | 5,703 | samples |
| ran; our contract draft was wrong, not the code | 50,332 | samples |
| ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) | 13,453 | samples |
| ran on a synthesized input | 125,936 | samples |
| unverified (harvested, no recorded run) | 83,091 | samples |
| samples with a behaviour fingerprint | 58,064 | samples |
| links whose sample has no function or class name | 0 | links |
| paper–sample links | 361,576 | links |
| links that are pointer only (Syntology does not serve that copy's text) | 142,682 | links |
| of which this paper's copy is cleared, but the stored copy of the same code that Syntology serves is not | 17,699 | links |
| links where this paper's copy is the one Syntology first harvested | 243,453 | links |
| links whose repository and path the harvester recorded for this paper | 100,333 | links |
| links whose repository and path come from a harvest plan row for this paper | 12,948 | links |
| links to identical code first harvested for another paper (this paper's copy not located) | 4,842 | links |
A sample is never promoted from "unverified" without a record: harvesting is cheap and execution is not. Statuses are computed at build time by the site's one mapping from the recorded level, report and contract verdict; the downloadable file carries the same statuses (download).