{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/code/format-data","entry":"format_data","source":"Syntology graph, per-sample; not an archive number","read_at":"2026-10-05T00:36:11+00:00","claim":"Names are grouped by exact entry-name string. Same-named routines are NOT asserted to be equivalent; 'ran' means executed on a synthesized fixture, not correctness. n_samples_ran = sum of by_status over every status except 'unverified' (ran_draft_wrong and ran_fixture are failures of Syntology's instrument, not of the code); n_papers_ran = papers with at least one such sample = n_papers_ran_no_instrument_failure (papers with a run with no instrument failure) + n_papers_ran_instrument_only (papers where every run was a failure of Syntology's instrument); n_samples_ran = n_samples_ran_checked (honoured, violated, no contract) + n_samples_instrument (failures of Syntology's instrument, not of the code).","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"n_papers":13,"n_papers_ran":12,"n_papers_ran_no_instrument_failure":4,"n_papers_ran_instrument_only":8,"units":"n_samples, n_samples_ran, n_samples_fingerprinted and by_status count distinct code bodies (code_sha256); n_places and n_places_pointer_only count places, one per (paper, code body) pair, which is also the unit of the samples list","n_samples":13,"n_samples_ran":12,"n_samples_constructed":0,"n_samples_ran_checked":4,"n_samples_instrument":8,"n_samples_unverified":1,"n_samples_fingerprinted":2,"n_places":13,"n_places_pointer_only":5,"n_places_pointer_only_link_open":0,"inline_ok_is":"inline_ok is true only when both licence gates allow Syntology to serve this code's text: this paper's link (inline_ok_link, from the licence of this paper's copy) and the stored copy of the same code that Syntology serves, which may have been first harvested from another repository. withheld_clause is Syntology's own wording of which gate keeps the text a pointer and why, and is null when inline_ok is true. This site shows no code text either way; n_places_pointer_only_link_open counts the places where this paper's link allows the text and the stored copy does not","by_status":{"ran_honours":0,"ran_violates":0,"ran_draft_wrong":8,"ran_fixture":0,"ran":4,"unverified":1},"key_notes":{"n_samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"syntology":{"atlas_url":null,"mcp":null,"mcp_per_sample":{"tool":"get_code","arguments_in":"samples[].mcp_get_code"},"developers":"https://syntology.ai/developers"},"samples":[{"arxiv_id":"2607.26062","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2607-26062","title":"Identifying implicit bias in LLM-based chat AI toward people with intellectual disabilities","date":null,"month_inferred_from_arxiv_id":"2026-07","title_source":"syntology","repo":"specialolympics/implicit_bias_llms","path":"code/format_data.py","file_url":"https://github.com/specialolympics/implicit_bias_llms/blob/HEAD/code/format_data.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GPL-3.0, as GitHub detects it on this paper's repository, specialolympics/implicit_bias_llms, is not in the permissive set Syntology serves inline","code_sha256":"c53374aaedeb79d4","code_sha256_prefix":"c53374aaedeb79d4","mcp_get_code":{"code_sha256":"c53374aaedeb79d4"}},{"arxiv_id":"2509.13760","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2509-13760","title":"Iterative Prompt Refinement for Safer Text-to-Image Generation","date":null,"month_inferred_from_arxiv_id":"2025-09","title_source":"syntology","repo":"ku-dmlab/IPR","path":"vl_sft.py","file_url":"https://github.com/ku-dmlab/IPR/blob/HEAD/vl_sft.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"affb4504d85ac0b7","code_sha256_prefix":"affb4504d85ac0b7","mcp_get_code":{"code_sha256":"affb4504d85ac0b7"}},{"arxiv_id":"2502.04463","arxiv_id_is_arxiv":true,"paper":"/paper/training-language-models-to-reason","title":"Training Language Models to Reason Efficiently","date":"2025-02-06","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"stardewxxx/o1-pruner","path":"o1_scripts/construct_dataset.py","file_url":"https://github.com/stardewxxx/o1-pruner/blob/HEAD/o1_scripts/construct_dataset.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"2818be86730dbfa8","code_sha256_prefix":"2818be86730dbfa8","mcp_get_code":{"code_sha256":"2818be86730dbfa8"}},{"arxiv_id":"2405.08460","arxiv_id_is_arxiv":true,"paper":"/paper/evaluating-llms-at-evaluating-temporal","title":"Is Your LLM Outdated? Evaluating LLMs at Temporal Generalization","date":"2024-05-14","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"freedomintelligence/freshbench","path":"script/format4score.py","file_url":"https://github.com/freedomintelligence/freshbench/blob/HEAD/script/format4score.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, freedomintelligence/freshbench when it was checked","code_sha256":"362cc4cc9069be45","code_sha256_prefix":"362cc4cc9069be45","mcp_get_code":{"code_sha256":"362cc4cc9069be45"}},{"arxiv_id":"2312.02852","arxiv_id_is_arxiv":true,"paper":"/paper/expert-guided-bayesian-optimisation-for-human","title":"Expert-guided Bayesian Optimisation for Human-in-the-loop Experimental Design of Known Systems","date":"2023-12-05","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"trsav/hitl-bo","path":"bo/utils.py","file_url":"https://github.com/trsav/hitl-bo/blob/HEAD/bo/utils.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"77467af29804f6b4","code_sha256_prefix":"77467af29804f6b4","mcp_get_code":{"code_sha256":"77467af29804f6b4"}},{"arxiv_id":"2311.09053","arxiv_id_is_arxiv":true,"paper":"/paper/assessing-knowledge-editing-in-language","title":"Assessing Knowledge Editing in Language Models via Relation Perspective","date":"2023-11-15","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"weiyifan1023/knowledge-edit-based-on-relation-perspective","path":"baselines/kn/knowledge_neurons/plot_pararel_results.py","file_url":"https://github.com/weiyifan1023/knowledge-edit-based-on-relation-perspective/blob/HEAD/baselines/kn/knowledge_neurons/plot_pararel_results.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, weiyifan1023/knowledge-edit-based-on-relation-perspective when it was checked","code_sha256":"2d71184c9abd680c","code_sha256_prefix":"2d71184c9abd680c","mcp_get_code":{"code_sha256":"2d71184c9abd680c"}},{"arxiv_id":"2305.19472","arxiv_id_is_arxiv":true,"paper":"/paper/plasma-making-small-language-models-better","title":"PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning","date":"2023-05-31","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"allenai/plasma","path":"verifier_guided_decoding/verifier_guided_generation.py","file_url":"https://github.com/allenai/plasma/blob/HEAD/verifier_guided_decoding/verifier_guided_generation.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"ec3022b867eaf47a","code_sha256_prefix":"ec3022b867eaf47a","mcp_get_code":{"code_sha256":"ec3022b867eaf47a"}},{"arxiv_id":"2210.17213","arxiv_id_is_arxiv":true,"paper":"/paper/deep-gaussian-process-based-multi-fidelity","title":"Deep Gaussian Process-based Multi-fidelity Bayesian Optimization for Simulated Chemical Reactors","date":"2022-10-31","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"optimal-pse-lab/pulsed-reactor-optimisation","path":"utils.py","file_url":"https://github.com/optimal-pse-lab/pulsed-reactor-optimisation/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"42c4b8839b4ec7a2","code_sha256_prefix":"42c4b8839b4ec7a2","mcp_get_code":{"code_sha256":"42c4b8839b4ec7a2"}},{"arxiv_id":"2205.06908","arxiv_id_is_arxiv":true,"paper":"/paper/neural-fly-enables-rapid-learning-for-agile","title":"Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds","date":"2022-05-13","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"aerorobotics/neural-fly","path":"utils.py","file_url":"https://github.com/aerorobotics/neural-fly/blob/HEAD/utils.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, aerorobotics/neural-fly when it was checked","code_sha256":"60fb21c65d52b0db","code_sha256_prefix":"60fb21c65d52b0db","mcp_get_code":{"code_sha256":"60fb21c65d52b0db"}},{"arxiv_id":"1802.07606","arxiv_id_is_arxiv":true,"paper":"/paper/ordered-preference-elicitation-strategies-for","title":"Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making","date":"2018-02-21","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"lmzintgraf/gp_pref_elicit","path":"gp_utilities/utils_data.py","file_url":"https://github.com/lmzintgraf/gp_pref_elicit/blob/HEAD/gp_utilities/utils_data.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"f76b913d68f5771b","code_sha256_prefix":"f76b913d68f5771b","mcp_get_code":{"code_sha256":"f76b913d68f5771b"}},{"arxiv_id":"1703.10135","arxiv_id_is_arxiv":true,"paper":"/paper/tacotron-towards-end-to-end-speech-synthesis","title":"Tacotron: Towards End-to-End Speech Synthesis","date":"2017-03-29","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"mozilla/TTS","path":"TTS/bin/train_tacotron.py","file_url":"https://github.com/mozilla/TTS/blob/HEAD/TTS/bin/train_tacotron.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MPL-2.0","inline_ok":false,"inline_ok_link":false,"withheld_clause":"MPL-2.0, as GitHub detects it on this paper's repository, mozilla/TTS, is not in the permissive set Syntology serves inline","code_sha256":"9df2870ead7d247b","code_sha256_prefix":"9df2870ead7d247b","mcp_get_code":{"code_sha256":"9df2870ead7d247b"}},{"arxiv_id":"1510.03055","arxiv_id_is_arxiv":true,"paper":"/paper/a-diversity-promoting-objective-function-for","title":"A Diversity-Promoting Objective Function for Neural Conversation Models","date":"2015-10-11","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"kennethjones17/chatbot","path":"1.py","file_url":"https://github.com/kennethjones17/chatbot/blob/HEAD/1.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"60443b1c88d3531b","code_sha256_prefix":"60443b1c88d3531b","mcp_get_code":{"code_sha256":"60443b1c88d3531b"}},{"arxiv_id":"1506.07285","arxiv_id_is_arxiv":true,"paper":"/paper/ask-me-anything-dynamic-memory-networks-for","title":"Ask Me Anything: Dynamic Memory Networks for Natural Language Processing","date":"2015-06-24","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"Asteur/someChatbot","path":"do_make_db_from_reddit.py","file_url":"https://github.com/Asteur/someChatbot/blob/HEAD/do_make_db_from_reddit.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"8d6964bc65ba3a5d","code_sha256_prefix":"8d6964bc65ba3a5d","mcp_get_code":{"code_sha256":"8d6964bc65ba3a5d"}}],"record_sha256":"67c1208484ab8720f11454d82820f7532f17e5f461cb691da0bb17c6b9e94501","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}