{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/code/task","entry":"Task","source":"Syntology graph, per-sample; not an archive number","read_at":"2026-10-05T00:36:11+00:00","claim":"Names are grouped by exact entry-name string. Same-named routines are NOT asserted to be equivalent; 'ran' means executed on a synthesized fixture, not correctness. n_samples_ran = sum of by_status over every status except 'unverified' (ran_draft_wrong and ran_fixture are failures of Syntology's instrument, not of the code); n_papers_ran = papers with at least one such sample = n_papers_ran_no_instrument_failure (papers with a run with no instrument failure) + n_papers_ran_instrument_only (papers where every run was a failure of Syntology's instrument); n_samples_ran = n_samples_ran_checked (honoured, violated, no contract) + n_samples_instrument (failures of Syntology's instrument, not of the code).","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"n_papers":17,"n_papers_ran":10,"n_papers_ran_no_instrument_failure":10,"n_papers_ran_instrument_only":0,"units":"n_samples, n_samples_ran, n_samples_fingerprinted and by_status count distinct code bodies (code_sha256); n_places and n_places_pointer_only count places, one per (paper, code body) pair, which is also the unit of the samples list","n_samples":17,"n_samples_ran":10,"n_samples_constructed":6,"n_samples_ran_checked":10,"n_samples_instrument":0,"n_samples_unverified":7,"n_samples_fingerprinted":0,"n_places":17,"n_places_pointer_only":7,"n_places_pointer_only_link_open":0,"inline_ok_is":"inline_ok is true only when both licence gates allow Syntology to serve this code's text: this paper's link (inline_ok_link, from the licence of this paper's copy) and the stored copy of the same code that Syntology serves, which may have been first harvested from another repository. withheld_clause is Syntology's own wording of which gate keeps the text a pointer and why, and is null when inline_ok is true. This site shows no code text either way; n_places_pointer_only_link_open counts the places where this paper's link allows the text and the stored copy does not","by_status":{"ran_honours":0,"ran_violates":0,"ran_draft_wrong":0,"ran_fixture":0,"ran":10,"unverified":7},"key_notes":{"n_samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"syntology":{"atlas_url":null,"mcp":null,"mcp_per_sample":{"tool":"get_code","arguments_in":"samples[].mcp_get_code"},"developers":"https://syntology.ai/developers"},"samples":[{"arxiv_id":"2607.25765","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2607-25765","title":"WorkSurface-Bench: Benchmarking Enterprise Agents on Multi-Surface Knowledge Routing","date":null,"month_inferred_from_arxiv_id":"2026-07","title_source":"syntology","repo":"haolpku/WorkSurface-Bench","path":"worksurface/derive_tasks.py","file_url":"https://github.com/haolpku/WorkSurface-Bench/blob/HEAD/worksurface/derive_tasks.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"50555611ef7241a5","code_sha256_prefix":"50555611ef7241a5","mcp_get_code":{"code_sha256":"50555611ef7241a5"}},{"arxiv_id":"2607.13411","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2607-13411","title":"Evaluating Frontier AI Agents as Autonomous Clinical Security Auditors","date":null,"month_inferred_from_arxiv_id":"2026-07","title_source":"syntology","repo":"MichaelEnny/clinical-ai-security-eval","path":"task/clinical_ai_security_eval.py","file_url":"https://github.com/MichaelEnny/clinical-ai-security-eval/blob/HEAD/task/clinical_ai_security_eval.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"2e1dd7706773f279","code_sha256_prefix":"2e1dd7706773f279","mcp_get_code":{"code_sha256":"2e1dd7706773f279"}},{"arxiv_id":"2606.24428","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2606-24428","title":"Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning","date":null,"month_inferred_from_arxiv_id":"2026-06","title_source":"syntology","repo":"shidingz/EDV","path":"src/edv/pipeline.py","file_url":"https://github.com/shidingz/EDV/blob/HEAD/src/edv/pipeline.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"7e6bf386dea53a1e","code_sha256_prefix":"7e6bf386dea53a1e","mcp_get_code":{"code_sha256":"7e6bf386dea53a1e"}},{"arxiv_id":"2603.24738","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2603-24738","title":"Decentralized Task Scheduling in Distributed Systems: A Deep Reinforcement Learning Approach","date":null,"month_inferred_from_arxiv_id":"2026-03","title_source":"syntology","repo":"danielbenniah/marl-distributed-scheduling","path":"marl_scheduler.py","file_url":"https://github.com/danielbenniah/marl-distributed-scheduling/blob/HEAD/marl_scheduler.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, danielbenniah/marl-distributed-scheduling when it was checked","code_sha256":"b22560173dde05bc","code_sha256_prefix":"b22560173dde05bc","mcp_get_code":{"code_sha256":"b22560173dde05bc"}},{"arxiv_id":"2407.19346","arxiv_id_is_arxiv":true,"paper":"/paper/polynomial-regression-as-a-task-for","title":"Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment","date":"2024-07-27","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"dtsip/in-context-learning","path":"src/tasks.py","file_url":"https://github.com/dtsip/in-context-learning/blob/HEAD/src/tasks.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"3e016983bf585f29","code_sha256_prefix":"3e016983bf585f29","mcp_get_code":{"code_sha256":"3e016983bf585f29"}},{"arxiv_id":"2402.04858","arxiv_id_is_arxiv":true,"paper":"/paper/codeit-self-improving-language-models-with","title":"CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay","date":"2024-02-07","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"Qualcomm-AI-research/codeit","path":"codeit/augment/genetic.py","file_url":"https://github.com/Qualcomm-AI-research/codeit/blob/HEAD/codeit/augment/genetic.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause-Clear","inline_ok":false,"inline_ok_link":false,"withheld_clause":"BSD-3-Clause-Clear, as GitHub detects it on this paper's repository, Qualcomm-AI-research/codeit, is not in the permissive set Syntology serves inline","code_sha256":"3215558dec21b76e","code_sha256_prefix":"3215558dec21b76e","mcp_get_code":{"code_sha256":"3215558dec21b76e"}},{"arxiv_id":"2310.15164","arxiv_id_is_arxiv":true,"paper":"/paper/linc-a-neurosymbolic-approach-for-logical","title":"LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers","date":"2023-10-23","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"benlipkin/linc","path":"eval/tasks/folio.py","file_url":"https://github.com/benlipkin/linc/blob/HEAD/eval/tasks/folio.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, benlipkin/linc when it was checked","code_sha256":"ed34cd2d0c6826c0","code_sha256_prefix":"ed34cd2d0c6826c0","mcp_get_code":{"code_sha256":"ed34cd2d0c6826c0"}},{"arxiv_id":"2310.04870","arxiv_id_is_arxiv":true,"paper":"/paper/lemur-integrating-large-language-models-in","title":"Lemur: Integrating Large Language Models in Automated Program Verification","date":"2023-10-07","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"wu-haoze/lemur-program-verification","path":"lemur/src/verifier.py","file_url":"https://github.com/wu-haoze/lemur-program-verification/blob/HEAD/lemur/src/verifier.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, wu-haoze/lemur-program-verification when it was checked","code_sha256":"45188c4494249615","code_sha256_prefix":"45188c4494249615","mcp_get_code":{"code_sha256":"45188c4494249615"}},{"arxiv_id":"2212.13738","arxiv_id_is_arxiv":true,"paper":"/paper/tempclr-temporal-alignment-representation","title":"TempCLR: Temporal Alignment Representation with Contrastive Learning","date":"2022-12-28","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"yyuncong/tempclr","path":"tempclr/tasks/alignlocaltask.py","file_url":"https://github.com/yyuncong/tempclr/blob/HEAD/tempclr/tasks/alignlocaltask.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"8e1461d6cc228441","code_sha256_prefix":"8e1461d6cc228441","mcp_get_code":{"code_sha256":"8e1461d6cc228441"}},{"arxiv_id":"2210.03629","arxiv_id_is_arxiv":true,"paper":"/paper/react-synergizing-reasoning-and-acting-in","title":"ReAct: Synergizing Reasoning and Acting in Language Models","date":"2022-10-06","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"sierra-research/tau-bench","path":"tau_bench/agents/chat_react_agent.py","file_url":"https://github.com/sierra-research/tau-bench/blob/HEAD/tau_bench/agents/chat_react_agent.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"c6504ae4ebf5bcab","code_sha256_prefix":"c6504ae4ebf5bcab","mcp_get_code":{"code_sha256":"c6504ae4ebf5bcab"}},{"arxiv_id":"2204.07705","arxiv_id_is_arxiv":true,"paper":"/paper/benchmarking-generalization-via-in-context","title":"Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks","date":"2022-04-16","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"zhaochenyang20/Prompt2Model-Self-Guide","path":"self-guide/utils/tasks.py","file_url":"https://github.com/zhaochenyang20/Prompt2Model-Self-Guide/blob/HEAD/self-guide/utils/tasks.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"588da051624b9d01","code_sha256_prefix":"588da051624b9d01","mcp_get_code":{"code_sha256":"588da051624b9d01"}},{"arxiv_id":"2203.13602","arxiv_id_is_arxiv":true,"paper":"/paper/zs4ie-a-toolkit-for-zero-shot-information","title":"ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizations","date":"2022-03-25","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"osainz59/Ask2Transformers","path":"a2t/tasks/base.py","file_url":"https://github.com/osainz59/Ask2Transformers/blob/HEAD/a2t/tasks/base.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"c9033c25890f4270","code_sha256_prefix":"c9033c25890f4270","mcp_get_code":{"code_sha256":"c9033c25890f4270"}},{"arxiv_id":"2102.03034","arxiv_id_is_arxiv":true,"paper":"/paper/hyperparameter-optimization-is-deceiving-us","title":"Hyperparameter Optimization Is Deceiving Us, and How to Stop It","date":"2021-02-05","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"pasta41/deception","path":"src/utils/utils.py","file_url":"https://github.com/pasta41/deception/blob/HEAD/src/utils/utils.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, pasta41/deception when it was checked","code_sha256":"1203fb727028792b","code_sha256_prefix":"1203fb727028792b","mcp_get_code":{"code_sha256":"1203fb727028792b"}},{"arxiv_id":"2003.02249","arxiv_id_is_arxiv":true,"paper":"/paper/jiant-a-software-toolkit-for-research-on","title":"jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models","date":"2020-03-04","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"nyu-mll/jiant","path":"jiant/proj/main/modeling/primary.py","file_url":"https://github.com/nyu-mll/jiant/blob/HEAD/jiant/proj/main/modeling/primary.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"3cc8da16c18465bd","code_sha256_prefix":"3cc8da16c18465bd","mcp_get_code":{"code_sha256":"3cc8da16c18465bd"}},{"arxiv_id":"1909.04630","arxiv_id_is_arxiv":true,"paper":"/paper/meta-learning-with-implicit-gradients","title":"Meta-Learning with Implicit Gradients","date":"2019-09-10","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"spiglerg/pyMeta","path":"pyMeta/metalearners/implicit_maml.py","file_url":"https://github.com/spiglerg/pyMeta/blob/HEAD/pyMeta/metalearners/implicit_maml.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, spiglerg/pyMeta when it was checked","code_sha256":"ca574f805c91d123","code_sha256_prefix":"ca574f805c91d123","mcp_get_code":{"code_sha256":"ca574f805c91d123"}},{"arxiv_id":"1810.04805","arxiv_id_is_arxiv":true,"paper":"/paper/bert-pre-training-of-deep-bidirectional","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","date":"2018-10-11","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"ABaldrati/MT-BERT","path":"model.py","file_url":"https://github.com/ABaldrati/MT-BERT/blob/HEAD/model.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, ABaldrati/MT-BERT when it was checked","code_sha256":"539b7f6e38fc9786","code_sha256_prefix":"539b7f6e38fc9786","mcp_get_code":{"code_sha256":"539b7f6e38fc9786"}},{"arxiv_id":"1703.03400","arxiv_id_is_arxiv":true,"paper":"/paper/model-agnostic-meta-learning-for-fast","title":"Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks","date":"2017-03-09","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"MaximeVandegar/Papers-in-100-Lines-of-Code","path":"Model_Agnostic_Meta_Learning_for_Fast_Adaptation_of_Deep_Networks/maml.py","file_url":"https://github.com/MaximeVandegar/Papers-in-100-Lines-of-Code/blob/HEAD/Model_Agnostic_Meta_Learning_for_Fast_Adaptation_of_Deep_Networks/maml.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"ec0fb012c0b639eb","code_sha256_prefix":"ec0fb012c0b639eb","mcp_get_code":{"code_sha256":"ec0fb012c0b639eb"}}],"record_sha256":"1e5b07a598d120a01966d52af0500473ea3403ca44a12a9dda1466d98c761740","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}