{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/code/serialize","entry":"serialize","source":"Syntology graph, per-sample; not an archive number","read_at":"2026-10-05T00:36:11+00:00","claim":"Names are grouped by exact entry-name string. Same-named routines are NOT asserted to be equivalent; 'ran' means executed on a synthesized fixture, not correctness. n_samples_ran = sum of by_status over every status except 'unverified' (ran_draft_wrong and ran_fixture are failures of Syntology's instrument, not of the code); n_papers_ran = papers with at least one such sample = n_papers_ran_no_instrument_failure (papers with a run with no instrument failure) + n_papers_ran_instrument_only (papers where every run was a failure of Syntology's instrument); n_samples_ran = n_samples_ran_checked (honoured, violated, no contract) + n_samples_instrument (failures of Syntology's instrument, not of the code).","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"n_papers":19,"n_papers_ran":19,"n_papers_ran_no_instrument_failure":11,"n_papers_ran_instrument_only":8,"units":"n_samples, n_samples_ran, n_samples_fingerprinted and by_status count distinct code bodies (code_sha256); n_places and n_places_pointer_only count places, one per (paper, code body) pair, which is also the unit of the samples list","n_samples":18,"n_samples_ran":18,"n_samples_constructed":0,"n_samples_ran_checked":11,"n_samples_instrument":7,"n_samples_unverified":0,"n_samples_fingerprinted":0,"n_places":19,"n_places_pointer_only":5,"n_places_pointer_only_link_open":0,"inline_ok_is":"inline_ok is true only when both licence gates allow Syntology to serve this code's text: this paper's link (inline_ok_link, from the licence of this paper's copy) and the stored copy of the same code that Syntology serves, which may have been first harvested from another repository. withheld_clause is Syntology's own wording of which gate keeps the text a pointer and why, and is null when inline_ok is true. This site shows no code text either way; n_places_pointer_only_link_open counts the places where this paper's link allows the text and the stored copy does not","by_status":{"ran_honours":0,"ran_violates":0,"ran_draft_wrong":7,"ran_fixture":0,"ran":11,"unverified":0},"key_notes":{"n_samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"syntology":{"atlas_url":null,"mcp":null,"mcp_per_sample":{"tool":"get_code","arguments_in":"samples[].mcp_get_code"},"developers":"https://syntology.ai/developers"},"samples":[{"arxiv_id":"2605.31464","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2605-31464","title":"GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization","date":null,"month_inferred_from_arxiv_id":"2026-05","title_source":"syntology","repo":"gpu-mode/kernelbot","path":"src/kernelbot/cogs/admin_cog.py","file_url":"https://github.com/gpu-mode/kernelbot/blob/HEAD/src/kernelbot/cogs/admin_cog.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub could not classify the licence of this paper's repository, gpu-mode/kernelbot (NOASSERTION)","code_sha256":"b14e84f4b6493413","code_sha256_prefix":"b14e84f4b6493413","mcp_get_code":{"code_sha256":"b14e84f4b6493413"}},{"arxiv_id":"2605.27621","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2605-27621","title":"Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution","date":null,"month_inferred_from_arxiv_id":"2026-05","title_source":"syntology","repo":"ybkim95/agent-scaling","path":"agent_scaling/logger.py","file_url":"https://github.com/ybkim95/agent-scaling/blob/HEAD/agent_scaling/logger.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"5bce9e2e8490b090","code_sha256_prefix":"5bce9e2e8490b090","mcp_get_code":{"code_sha256":"5bce9e2e8490b090"}},{"arxiv_id":"2603.02913","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2603-02913","title":"Eliciting Numerical Predictive Distributions of LLMs Without Autoregression","date":null,"month_inferred_from_arxiv_id":"2026-03","title_source":"syntology","repo":"kasia-kobalczyk/guess_llm","path":"src/guess_llm/llm_utils/llm_no_scaling.py","file_url":"https://github.com/kasia-kobalczyk/guess_llm/blob/HEAD/src/guess_llm/llm_utils/llm_no_scaling.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub could not classify the licence of this paper's repository, kasia-kobalczyk/guess_llm (NOASSERTION)","code_sha256":"2b7966b3ab732fd3","code_sha256_prefix":"2b7966b3ab732fd3","mcp_get_code":{"code_sha256":"2b7966b3ab732fd3"}},{"arxiv_id":"2601.21665","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2601-21665","title":"AdaptBPE: From General Purpose to Specialized Tokenizers","date":null,"month_inferred_from_arxiv_id":"2026-01","title_source":"syntology","repo":"vijini/Adapt-BPE","path":"apply_merges_with_true_counts.py","file_url":"https://github.com/vijini/Adapt-BPE/blob/HEAD/apply_merges_with_true_counts.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, vijini/Adapt-BPE when it was checked","code_sha256":"6dcf170f38f4cd97","code_sha256_prefix":"6dcf170f38f4cd97","mcp_get_code":{"code_sha256":"6dcf170f38f4cd97"}},{"arxiv_id":"2506.08249","arxiv_id_is_arxiv":true,"paper":"/paper/2506-08249","title":"RADAR: Benchmarking Language Models on Imperfect Tabular Data","date":"2025-06-09","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"kenqgu/radar","path":"radar/logger.py","file_url":"https://github.com/kenqgu/radar/blob/HEAD/radar/logger.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"5bce9e2e8490b090","code_sha256_prefix":"5bce9e2e8490b090","mcp_get_code":{"code_sha256":"5bce9e2e8490b090"}},{"arxiv_id":"2504.21034","arxiv_id_is_arxiv":true,"paper":"/paper/saga-a-security-architecture-for-governing-ai","title":"SAGA: A Security Architecture for Governing AI Agentic Systems","date":"2025-04-27","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"gsiros/saga","path":"saga/agent.py","file_url":"https://github.com/gsiros/saga/blob/HEAD/saga/agent.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"cb05405286e7bc8c","code_sha256_prefix":"cb05405286e7bc8c","mcp_get_code":{"code_sha256":"cb05405286e7bc8c"}},{"arxiv_id":"2503.14434","arxiv_id_is_arxiv":true,"paper":"/paper/llm-fe-automated-feature-engineering-for","title":"LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers","date":"2025-03-18","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"nikhilsab/llmfe","path":"utils.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"4d9bef27ef967fbe","code_sha256_prefix":"4d9bef27ef967fbe","mcp_get_code":{"code_sha256":"4d9bef27ef967fbe"}},{"arxiv_id":"2404.09491","arxiv_id_is_arxiv":true,"paper":"/paper/large-language-models-can-automatically","title":"Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning","date":"2024-04-15","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"sungwon-han/featllm","path":"utils.py","file_url":"https://github.com/sungwon-han/featllm/blob/HEAD/utils.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, sungwon-han/featllm when it was checked","code_sha256":"36cfd9635b49c74a","code_sha256_prefix":"36cfd9635b49c74a","mcp_get_code":{"code_sha256":"36cfd9635b49c74a"}},{"arxiv_id":"2402.14672","arxiv_id_is_arxiv":true,"paper":"/paper/middleware-for-llms-tools-are-instrumental","title":"Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments","date":"2024-02-22","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"OSU-NLP-Group/Middleware","path":"src/utils.py","file_url":"https://github.com/OSU-NLP-Group/Middleware/blob/HEAD/src/utils.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, OSU-NLP-Group/Middleware when it was checked","code_sha256":"22a1fe0dc1bdac57","code_sha256_prefix":"22a1fe0dc1bdac57","mcp_get_code":{"code_sha256":"22a1fe0dc1bdac57"}},{"arxiv_id":"2308.03688","arxiv_id_is_arxiv":true,"paper":"/paper/agentbench-evaluating-llms-as-agents","title":"AgentBench: Evaluating LLMs as Agents","date":"2023-08-07","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"THUDM/AgentBench","path":"src/utils/others.py","file_url":"https://github.com/THUDM/AgentBench/blob/HEAD/src/utils/others.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"02ff76b8a757ac06","code_sha256_prefix":"02ff76b8a757ac06","mcp_get_code":{"code_sha256":"02ff76b8a757ac06"}},{"arxiv_id":"2204.02574","arxiv_id_is_arxiv":true,"paper":"/paper/focalclick-towards-practical-interactive","title":"FocalClick: Towards Practical Interactive Image Segmentation","date":"2022-04-06","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"XavierCHEN34/ClickSEG","path":"isegm/model/is_hrnet_model.py","file_url":"https://github.com/XavierCHEN34/ClickSEG/blob/HEAD/isegm/model/is_hrnet_model.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"a4cd31544f3ea2b5","code_sha256_prefix":"a4cd31544f3ea2b5","mcp_get_code":{"code_sha256":"a4cd31544f3ea2b5"}},{"arxiv_id":"2203.02025","arxiv_id_is_arxiv":true,"paper":"/paper/online-balanced-experimental-design","title":"Online Balanced Experimental Design","date":null,"month_inferred_from_arxiv_id":"2022-03","title_source":"archive","repo":"ddimmery/balancer-package","path":"src/bwd/serialization.py","file_url":"https://github.com/ddimmery/balancer-package/blob/HEAD/src/bwd/serialization.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"82a33e621de5d9a7","code_sha256_prefix":"82a33e621de5d9a7","mcp_get_code":{"code_sha256":"82a33e621de5d9a7"}},{"arxiv_id":"2202.00666","arxiv_id_is_arxiv":true,"paper":"/paper/typical-decoding-for-natural-language","title":"Locally Typical Sampling","date":"2022-02-01","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"HomebrewNLP/HomebrewNLP-Jax","path":"src/context.py","file_url":"https://github.com/HomebrewNLP/HomebrewNLP-Jax/blob/HEAD/src/context.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"417d7c14445e03e0","code_sha256_prefix":"417d7c14445e03e0","mcp_get_code":{"code_sha256":"417d7c14445e03e0"}},{"arxiv_id":"2111.09453","arxiv_id_is_arxiv":true,"paper":"/paper/robertuito-a-pre-trained-language-model-for","title":"RoBERTuito: a pre-trained language model for social media text in Spanish","date":"2021-11-18","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"pysentimiento/robertuito","path":"finetune_vs_scratch/context_hate.py","file_url":"https://github.com/pysentimiento/robertuito/blob/HEAD/finetune_vs_scratch/context_hate.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"6bd359037885bc59","code_sha256_prefix":"6bd359037885bc59","mcp_get_code":{"code_sha256":"6bd359037885bc59"}},{"arxiv_id":"2107.08924","arxiv_id_is_arxiv":true,"paper":"/paper/epistemic-neural-networks-1","title":"Epistemic Neural Networks","date":"2021-07-19","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"google-deepmind/enn","path":"enn/loggers.py","file_url":"https://github.com/google-deepmind/enn/blob/HEAD/enn/loggers.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"73a9d7f8f0ed39a0","code_sha256_prefix":"73a9d7f8f0ed39a0","mcp_get_code":{"code_sha256":"73a9d7f8f0ed39a0"}},{"arxiv_id":"2003.09040","arxiv_id_is_arxiv":true,"paper":"/paper/tf-coder-program-synthesis-for-tensor","title":"TF-Coder: Program Synthesis for Tensor Manipulations","date":"2020-03-19","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"google-research/tensorflow-coder","path":"tf_coder_colab_logging/serialization.py","file_url":"https://github.com/google-research/tensorflow-coder/blob/HEAD/tf_coder_colab_logging/serialization.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"6363dbc5c6d4760f","code_sha256_prefix":"6363dbc5c6d4760f","mcp_get_code":{"code_sha256":"6363dbc5c6d4760f"}},{"arxiv_id":"1612.01627","arxiv_id_is_arxiv":true,"paper":"/paper/sequential-matching-network-a-new","title":"Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots","date":"2016-12-06","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"yangliuy/NeuralResponseRanking","path":"matchzoo/losses/rank_losses.py","file_url":"https://github.com/yangliuy/NeuralResponseRanking/blob/HEAD/matchzoo/losses/rank_losses.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"3d063b0c90233541","code_sha256_prefix":"3d063b0c90233541","mcp_get_code":{"code_sha256":"3d063b0c90233541"}},{"arxiv_id":"1506.03694","arxiv_id_is_arxiv":true,"paper":"/paper/learning-language-through-pictures","title":"Learning language through pictures","date":"2015-06-11","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"gchrupala/imaginet","path":"utils.py","file_url":"https://github.com/gchrupala/imaginet/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"2e2dbf986438cf7a","code_sha256_prefix":"2e2dbf986438cf7a","mcp_get_code":{"code_sha256":"2e2dbf986438cf7a"}},{"arxiv_id":"2024.findings-emnlp.815","arxiv_id_is_arxiv":false,"paper":null,"title":"ACL Anthology id 2024.findings-emnlp.815","date":null,"month_inferred_from_arxiv_id":null,"title_source":null,"repo":"behavioral-data/BLADE","path":"blade_bench/logger.py","file_url":"https://github.com/behavioral-data/BLADE/blob/HEAD/blade_bench/logger.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"8daabb19cdef1e76","code_sha256_prefix":"8daabb19cdef1e76","mcp_get_code":{"code_sha256":"8daabb19cdef1e76"}}],"record_sha256":"508f81768f1553f5b1e872327015190790cd0a613647f87c2cccd12acd61fd44","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}