{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/code/get-metrics","entry":"get_metrics","source":"Syntology graph, per-sample; not an archive number","read_at":"2026-10-05T00:36:11+00:00","claim":"Names are grouped by exact entry-name string. Same-named routines are NOT asserted to be equivalent; 'ran' means executed on a synthesized fixture, not correctness. n_samples_ran = sum of by_status over every status except 'unverified' (ran_draft_wrong and ran_fixture are failures of Syntology's instrument, not of the code); n_papers_ran = papers with at least one such sample = n_papers_ran_no_instrument_failure (papers with a run with no instrument failure) + n_papers_ran_instrument_only (papers where every run was a failure of Syntology's instrument); n_samples_ran = n_samples_ran_checked (honoured, violated, no contract) + n_samples_instrument (failures of Syntology's instrument, not of the code).","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"n_papers":46,"n_papers_ran":36,"n_papers_ran_no_instrument_failure":23,"n_papers_ran_instrument_only":13,"units":"n_samples, n_samples_ran, n_samples_fingerprinted and by_status count distinct code bodies (code_sha256); n_places and n_places_pointer_only count places, one per (paper, code body) pair, which is also the unit of the samples list","n_samples":42,"n_samples_ran":32,"n_samples_constructed":0,"n_samples_ran_checked":21,"n_samples_instrument":11,"n_samples_unverified":10,"n_samples_fingerprinted":8,"n_places":47,"n_places_pointer_only":22,"n_places_pointer_only_link_open":0,"inline_ok_is":"inline_ok is true only when both licence gates allow Syntology to serve this code's text: this paper's link (inline_ok_link, from the licence of this paper's copy) and the stored copy of the same code that Syntology serves, which may have been first harvested from another repository. withheld_clause is Syntology's own wording of which gate keeps the text a pointer and why, and is null when inline_ok is true. This site shows no code text either way; n_places_pointer_only_link_open counts the places where this paper's link allows the text and the stored copy does not","by_status":{"ran_honours":1,"ran_violates":0,"ran_draft_wrong":6,"ran_fixture":5,"ran":20,"unverified":10},"key_notes":{"n_samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"syntology":{"atlas_url":null,"mcp":null,"mcp_per_sample":{"tool":"get_code","arguments_in":"samples[].mcp_get_code"},"developers":"https://syntology.ai/developers"},"samples":[{"arxiv_id":"2608.03926","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2608-03926","title":"PRISM: Powerful Time Series to Image (TS2I) Representations for Multivariate Anomaly Detection","date":null,"month_inferred_from_arxiv_id":"2026-08","title_source":"syntology","repo":"Smendowski/PRISM","path":"TSB_AD/evaluation/metrics.py","file_url":"https://github.com/Smendowski/PRISM/blob/HEAD/TSB_AD/evaluation/metrics.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"7c220f2a618454a1","code_sha256_prefix":"7c220f2a618454a1","mcp_get_code":{"code_sha256":"7c220f2a618454a1"}},{"arxiv_id":"2606.20443","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2606-20443","title":"Topological Data Analysis for High-Dimensional Dynamic Process Monitoring","date":null,"month_inferred_from_arxiv_id":"2026-06","title_source":"syntology","repo":"zavalab/ML","path":"Asthma/src/result_utils.py","file_url":"https://github.com/zavalab/ML/blob/HEAD/Asthma/src/result_utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, zavalab/ML when it was checked","code_sha256":"7b0ae182cdf4004f","code_sha256_prefix":"7b0ae182cdf4004f","mcp_get_code":{"code_sha256":"7b0ae182cdf4004f"}},{"arxiv_id":"2605.24693","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2605-24693","title":"CP-Agent: A Calibrated Risk-Controlled Agent for Feedback-Driven Competitive Programming","date":null,"month_inferred_from_arxiv_id":"2026-05","title_source":"syntology","repo":"NineAbyss/CP-Agent","path":"LiveCodeBench-Pro/get_metrics.py","file_url":"https://github.com/NineAbyss/CP-Agent/blob/HEAD/LiveCodeBench-Pro/get_metrics.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, NineAbyss/CP-Agent when it was checked","code_sha256":"787a0fde31eb9f2e","code_sha256_prefix":"787a0fde31eb9f2e","mcp_get_code":{"code_sha256":"787a0fde31eb9f2e"}},{"arxiv_id":"2502.00593","arxiv_id_is_arxiv":true,"paper":"/paper/dominated-novelty-search-rethinking-local","title":"Dominated Novelty Search: Rethinking Local Competition in Quality-Diversity","date":"2025-02-01","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"adaptive-intelligent-robotics/dominated-novelty-search","path":"utils.py","file_url":"https://github.com/adaptive-intelligent-robotics/dominated-novelty-search/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"895137189de3e4c2","code_sha256_prefix":"895137189de3e4c2","mcp_get_code":{"code_sha256":"895137189de3e4c2"}},{"arxiv_id":"2410.22938","arxiv_id_is_arxiv":true,"paper":"/paper/difflight-a-partial-rewards-conditioned","title":"DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data","date":"2024-10-30","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"lokol5579/DiffLight-release","path":"summary.py","file_url":"https://github.com/lokol5579/DiffLight-release/blob/HEAD/summary.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GPL-3.0, as GitHub detects it on this paper's repository, lokol5579/DiffLight-release, is not in the permissive set Syntology serves inline","code_sha256":"89ca501b3a3e10c7","code_sha256_prefix":"89ca501b3a3e10c7","mcp_get_code":{"code_sha256":"89ca501b3a3e10c7"}},{"arxiv_id":"2410.17564","arxiv_id_is_arxiv":true,"paper":"/paper/disengcd-a-meta-multigraph-assisted","title":"DisenGCD: A Meta Multigraph-assisted Disentangled Graph Learning Framework for Cognitive Diagnosis","date":"2024-10-23","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"bimk/intelligent-education","path":"CTNCM/Train_for_CTNCM.py","file_url":"https://github.com/bimk/intelligent-education/blob/HEAD/CTNCM/Train_for_CTNCM.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, bimk/intelligent-education when it was checked","code_sha256":"9bcdc27c4e7a519c","code_sha256_prefix":"9bcdc27c4e7a519c","mcp_get_code":{"code_sha256":"9bcdc27c4e7a519c"}},{"arxiv_id":"2407.11052","arxiv_id_is_arxiv":true,"paper":"/paper/revisiting-benchmarking-and-understanding-1","title":"Revisiting, Benchmarking and Understanding Unsupervised Graph Domain Adaptation","date":"2024-07-09","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"rynewu224/GraphDA","path":"utils.py","file_url":"https://github.com/rynewu224/GraphDA/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, rynewu224/GraphDA when it was checked","code_sha256":"7957f349f88f21ca","code_sha256_prefix":"7957f349f88f21ca","mcp_get_code":{"code_sha256":"7957f349f88f21ca"}},{"arxiv_id":"2407.08280","arxiv_id_is_arxiv":true,"paper":"/paper/wayvescenes101-a-dataset-and-benchmark-for","title":"WayveScenes101: A Dataset and Benchmark for Novel View Synthesis in Autonomous Driving","date":"2024-07-11","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"wayveai/wayve_scenes","path":"src/wayve_scenes/utils/metrics.py","file_url":"https://github.com/wayveai/wayve_scenes/blob/HEAD/src/wayve_scenes/utils/metrics.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"f43b4b442a26ff4d","code_sha256_prefix":"f43b4b442a26ff4d","mcp_get_code":{"code_sha256":"f43b4b442a26ff4d"}},{"arxiv_id":"2404.01933","arxiv_id_is_arxiv":true,"paper":"/paper/prego-online-mistake-detection-in-procedural","title":"PREGO: online mistake detection in PRocedural EGOcentric videos","date":"2024-04-02","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"aleflabo/PREGO","path":"step_anticipation/src/models/llama_meta.py","file_url":"https://github.com/aleflabo/PREGO/blob/HEAD/step_anticipation/src/models/llama_meta.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"ded3bc4e3cdab571","code_sha256_prefix":"ded3bc4e3cdab571","mcp_get_code":{"code_sha256":"ded3bc4e3cdab571"}},{"arxiv_id":"2403.18301","arxiv_id_is_arxiv":true,"paper":"/paper/selective-mixup-fine-tuning-for-optimizing","title":"Selective Mixup Fine-Tuning for Optimizing Non-Decomposable Objectives","date":"2024-03-27","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"val-iisc/selmix","path":"finetuning/selmix_ssl.py","file_url":"https://github.com/val-iisc/selmix/blob/HEAD/finetuning/selmix_ssl.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, val-iisc/selmix when it was checked","code_sha256":"efc9682fffe68450","code_sha256_prefix":"efc9682fffe68450","mcp_get_code":{"code_sha256":"efc9682fffe68450"}},{"arxiv_id":"2403.12393","arxiv_id_is_arxiv":true,"paper":"/paper/dr3-ask-large-language-models-not-to-give-off","title":"Dr3: Ask Large Language Models Not to Give Off-Topic Answers in Open Domain Multi-Hop Question Answering","date":"2024-03-19","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"gy915/dr3","path":"Dr3/metrics.py","file_url":"https://github.com/gy915/dr3/blob/HEAD/Dr3/metrics.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, gy915/dr3 when it was checked","code_sha256":"1dbe229d47cf737d","code_sha256_prefix":"1dbe229d47cf737d","mcp_get_code":{"code_sha256":"1dbe229d47cf737d"}},{"arxiv_id":"2311.15367","arxiv_id_is_arxiv":true,"paper":"/paper/batchnorm-based-weakly-supervised-video","title":"BatchNorm-based Weakly Supervised Video Anomaly Detection","date":"2023-11-26","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"cool-xuan/bn-wvad","path":"infer.py","file_url":"https://github.com/cool-xuan/bn-wvad/blob/HEAD/infer.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, cool-xuan/bn-wvad when it was checked","code_sha256":"94deaa29eb5d8e90","code_sha256_prefix":"94deaa29eb5d8e90","mcp_get_code":{"code_sha256":"94deaa29eb5d8e90"}},{"arxiv_id":"2311.01007","arxiv_id_is_arxiv":true,"paper":"/paper/effective-human-ai-teams-via-learned-natural-1","title":"Effective Human-AI Teams via Learned Natural Language Rules and Onboarding","date":"2023-11-02","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"clinicalml/onboarding_human_ai","path":"src/teacher_methods/teacher_human.py","file_url":"https://github.com/clinicalml/onboarding_human_ai/blob/HEAD/src/teacher_methods/teacher_human.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, clinicalml/onboarding_human_ai when it was checked","code_sha256":"83e80b8b56bd781e","code_sha256_prefix":"83e80b8b56bd781e","mcp_get_code":{"code_sha256":"83e80b8b56bd781e"}},{"arxiv_id":"2310.12024","arxiv_id_is_arxiv":true,"paper":"/paper/core-a-few-shot-company-relation","title":"CORE: A Few-Shot Company Relation Classification Dataset for Robust Domain Adaptation","date":"2023-10-18","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"pnborchert/CORE","path":"benchmark/fs_run.py","file_url":"https://github.com/pnborchert/CORE/blob/HEAD/benchmark/fs_run.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, pnborchert/CORE when it was checked","code_sha256":"7baed95039e8f5e8","code_sha256_prefix":"7baed95039e8f5e8","mcp_get_code":{"code_sha256":"7baed95039e8f5e8"}},{"arxiv_id":"2310.01180","arxiv_id_is_arxiv":true,"paper":"/paper/evolutionary-neural-architecture-search-for-1","title":"Evolutionary Neural Architecture Search for Transformer in Knowledge Tracing","date":"2023-10-02","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"devilyangs/enas-kt","path":"utils.py","file_url":"https://github.com/devilyangs/enas-kt/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, devilyangs/enas-kt when it was checked","code_sha256":"73aac25ea9f0d376","code_sha256_prefix":"73aac25ea9f0d376","mcp_get_code":{"code_sha256":"73aac25ea9f0d376"}},{"arxiv_id":"2309.15098","arxiv_id_is_arxiv":true,"paper":"/paper/attention-satisfies-a-constraint-satisfaction","title":"Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models","date":"2023-09-26","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"microsoft/mechanistic-error-probe","path":"main_probe.py","file_url":"https://github.com/microsoft/mechanistic-error-probe/blob/HEAD/main_probe.py","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub could not classify the licence of this paper's repository, microsoft/mechanistic-error-probe (NOASSERTION)","code_sha256":"4d46774ab77c1ff3","code_sha256_prefix":"4d46774ab77c1ff3","mcp_get_code":{"code_sha256":"4d46774ab77c1ff3"}},{"arxiv_id":"2309.10790","arxiv_id_is_arxiv":true,"paper":null,"title":"arXiv:2309.10790","date":null,"month_inferred_from_arxiv_id":"2023-09","title_source":null,"repo":"csmile-1006/ARP","path":"arp_dt/utils.py","file_url":"https://github.com/csmile-1006/ARP/blob/HEAD/arp_dt/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"f051af6f08e6125a","code_sha256_prefix":"f051af6f08e6125a","mcp_get_code":{"code_sha256":"f051af6f08e6125a"}},{"arxiv_id":"2308.13418","arxiv_id_is_arxiv":true,"paper":"/paper/nougat-neural-optical-understanding-for","title":"Nougat: Neural Optical Understanding for Academic Documents","date":"2023-08-25","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"facebookresearch/nougat","path":"nougat/metrics.py","file_url":"https://github.com/facebookresearch/nougat/blob/HEAD/nougat/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"5d00412628beeb71","code_sha256_prefix":"5d00412628beeb71","mcp_get_code":{"code_sha256":"5d00412628beeb71"}},{"arxiv_id":"2308.07192","arxiv_id_is_arxiv":true,"paper":"/paper/gsasrec-reducing-overconfidence-in-sequential","title":"gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling","date":"2023-08-14","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"asash/gsasrec","path":"evaluation/analyze_experiment_in_progress.py","file_url":"https://github.com/asash/gsasrec/blob/HEAD/evaluation/analyze_experiment_in_progress.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"455c95e4d2eb8300","code_sha256_prefix":"455c95e4d2eb8300","mcp_get_code":{"code_sha256":"455c95e4d2eb8300"}},{"arxiv_id":"2306.12327","arxiv_id_is_arxiv":true,"paper":"/paper/learning-the-galaxy-environment-connection","title":"Learning the galaxy-environment connection with graph neural networks","date":null,"month_inferred_from_arxiv_id":"2023-06","title_source":"archive","repo":"jwuphysics/halo-gnns","path":"src/painting_galaxies.py","file_url":"https://github.com/jwuphysics/halo-gnns/blob/HEAD/src/painting_galaxies.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, jwuphysics/halo-gnns when it was checked","code_sha256":"499bf5ac75ad6a5e","code_sha256_prefix":"499bf5ac75ad6a5e","mcp_get_code":{"code_sha256":"499bf5ac75ad6a5e"}},{"arxiv_id":"2306.07111","arxiv_id_is_arxiv":true,"paper":"/paper/linear-classifier-an-often-forgotten-baseline","title":"Linear Classifier: An Often-Forgotten Baseline for Text Classification","date":"2023-06-12","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"ASUS-AICS/LibMultiLabel","path":"libmultilabel/linear/metrics.py","file_url":"https://github.com/ASUS-AICS/LibMultiLabel/blob/HEAD/libmultilabel/linear/metrics.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"8ed78a63461e3ac9","code_sha256_prefix":"8ed78a63461e3ac9","mcp_get_code":{"code_sha256":"8ed78a63461e3ac9"}},{"arxiv_id":"2304.14738","arxiv_id_is_arxiv":true,"paper":"/paper/cost-sensitive-self-training-for-optimizing","title":"Cost-Sensitive Self-Training for Optimizing Non-Decomposable Metrics","date":"2023-04-28","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"val-iisc/costsensitiveselftraining","path":"models/fixmatch/fixmatchCSST.py","file_url":"https://github.com/val-iisc/costsensitiveselftraining/blob/HEAD/models/fixmatch/fixmatchCSST.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, val-iisc/costsensitiveselftraining when it was checked","code_sha256":"ef03e1684bfdfe80","code_sha256_prefix":"ef03e1684bfdfe80","mcp_get_code":{"code_sha256":"ef03e1684bfdfe80"}},{"arxiv_id":"2304.09048","arxiv_id_is_arxiv":true,"paper":"/paper/codekgc-code-language-model-for-generative","title":"CodeKGC: Code Language Model for Generative Knowledge Graph Construction","date":"2023-04-18","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"zjunlp/DeepKE","path":"src/deepke/triple_extraction/PRGC/evaluate.py","file_url":"https://github.com/zjunlp/DeepKE/blob/HEAD/src/deepke/triple_extraction/PRGC/evaluate.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"626bdc34ef9914c2","code_sha256_prefix":"626bdc34ef9914c2","mcp_get_code":{"code_sha256":"626bdc34ef9914c2"}},{"arxiv_id":"2212.07796","arxiv_id_is_arxiv":true,"paper":"/paper/crepe-can-vision-language-foundation-models","title":"CREPE: Can Vision-Language Foundation Models Reason Compositionally?","date":"2022-12-13","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"raivnlab/crepe","path":"crepe_eval_utils.py","file_url":"https://github.com/raivnlab/crepe/blob/HEAD/crepe_eval_utils.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, raivnlab/crepe when it was checked","code_sha256":"ac38eb0d52769ad0","code_sha256_prefix":"ac38eb0d52769ad0","mcp_get_code":{"code_sha256":"ac38eb0d52769ad0"}},{"arxiv_id":"2211.09527","arxiv_id_is_arxiv":true,"paper":"/paper/ignore-previous-prompt-attack-techniques-for","title":"Ignore Previous Prompt: Attack Techniques For Language Models","date":"2022-11-17","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"agencyenterprise/PromptInject","path":"promptinject/scoring.py","file_url":"https://github.com/agencyenterprise/PromptInject/blob/HEAD/promptinject/scoring.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"515eba622dac0d78","code_sha256_prefix":"515eba622dac0d78","mcp_get_code":{"code_sha256":"515eba622dac0d78"}},{"arxiv_id":"2210.13431","arxiv_id_is_arxiv":true,"paper":"/paper/instruction-following-agents-with-jointly-pre","title":"Instruction-Following Agents with Multimodal Transformer","date":"2022-10-24","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"lhao499/instructrl","path":"instructrl/utils.py","file_url":"https://github.com/lhao499/instructrl/blob/HEAD/instructrl/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"f051af6f08e6125a","code_sha256_prefix":"f051af6f08e6125a","mcp_get_code":{"code_sha256":"f051af6f08e6125a"}},{"arxiv_id":"2210.10709","arxiv_id_is_arxiv":true,"paper":"/paper/schema-aware-reference-as-prompt-improves","title":"Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction","date":"2022-10-19","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"zjunlp/RAP","path":"BaseModel/PRGC/evaluate.py","file_url":"https://github.com/zjunlp/RAP/blob/HEAD/BaseModel/PRGC/evaluate.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"626bdc34ef9914c2","code_sha256_prefix":"626bdc34ef9914c2","mcp_get_code":{"code_sha256":"626bdc34ef9914c2"}},{"arxiv_id":"2210.03501","arxiv_id_is_arxiv":true,"paper":"/paper/towards-multi-modal-sarcasm-detection-via","title":"Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge Enhancement","date":"2022-10-07","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"less-and-less-bugs/hkemodel","path":"utils/compute_scores.py","file_url":"https://github.com/less-and-less-bugs/hkemodel/blob/HEAD/utils/compute_scores.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"249b3fa8193d045c","code_sha256_prefix":"249b3fa8193d045c","mcp_get_code":{"code_sha256":"249b3fa8193d045c"}},{"arxiv_id":"2210.03475","arxiv_id_is_arxiv":true,"paper":"/paper/winner-takes-it-all-training-performant-rl-1","title":"Winner Takes It All: Training Performant RL Populations for Combinatorial Optimization","date":"2022-10-07","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"instadeepai/poppy","path":"poppy/utils/metrics.py","file_url":"https://github.com/instadeepai/poppy/blob/HEAD/poppy/utils/metrics.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"05089f65276ed271","code_sha256_prefix":"05089f65276ed271","mcp_get_code":{"code_sha256":"05089f65276ed271"}},{"arxiv_id":"2207.12201","arxiv_id_is_arxiv":true,"paper":"/paper/calibrated-one-class-classification-for","title":"Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection","date":"2022-07-25","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"xuhongzuo/couta","path":"src/utils_eval.py","file_url":"https://github.com/xuhongzuo/couta/blob/HEAD/src/utils_eval.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"ad6de4f08952b158","code_sha256_prefix":"ad6de4f08952b158","mcp_get_code":{"code_sha256":"ad6de4f08952b158"}},{"arxiv_id":"2207.07483","arxiv_id_is_arxiv":true,"paper":"/paper/a-systematic-review-and-replicability-study","title":"A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation","date":"2022-07-15","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":null,"path":"","file_url":null,"status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"inline_ok_link":false,"withheld_clause":"this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it","code_sha256":"455c95e4d2eb8300","code_sha256_prefix":"455c95e4d2eb8300","mcp_get_code":{"code_sha256":"455c95e4d2eb8300"}},{"arxiv_id":"2206.01934","arxiv_id_is_arxiv":true,"paper":"/paper/stochastic-multiple-target-sampling-gradient","title":"Stochastic Multiple Target Sampling Gradient Descent","date":"2022-06-04","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"isl-org/MultiObjectiveOptimization","path":"multi_task/metrics.py","file_url":"https://github.com/isl-org/MultiObjectiveOptimization/blob/HEAD/multi_task/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"7adcc62e674f9a40","code_sha256_prefix":"7adcc62e674f9a40","mcp_get_code":{"code_sha256":"7adcc62e674f9a40"}},{"arxiv_id":"2204.04046","arxiv_id_is_arxiv":true,"paper":"/paper/kcd-knowledge-walks-and-textual-cues-enhanced-1","title":"KCD: Knowledge Walks and Textual Cues Enhanced Political Perspective Detection in News Media","date":"2022-04-08","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":null,"path":"","file_url":null,"status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"inline_ok_link":false,"withheld_clause":"this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it","code_sha256":"2b09b7b0191e6f69","code_sha256_prefix":"2b09b7b0191e6f69","mcp_get_code":{"code_sha256":"2b09b7b0191e6f69"}},{"arxiv_id":"2201.05057","arxiv_id_is_arxiv":true,"paper":"/paper/on-adversarial-robustness-of-trajectory","title":"On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles","date":"2022-01-13","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"zqzqz/AdvTrajectoryPrediction","path":"prediction/attack/gradient.py","file_url":"https://github.com/zqzqz/AdvTrajectoryPrediction/blob/HEAD/prediction/attack/gradient.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, zqzqz/AdvTrajectoryPrediction when it was checked","code_sha256":"19fe48e11c04f16f","code_sha256_prefix":"19fe48e11c04f16f","mcp_get_code":{"code_sha256":"19fe48e11c04f16f"}},{"arxiv_id":"2109.06024","arxiv_id_is_arxiv":true,"paper":"/paper/formalizing-and-estimating-distribution","title":"Formalizing and Estimating Distribution Inference Risks","date":"2021-09-13","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"iamgroot42/form_est_dist_risks","path":"botnets/model_utils.py","file_url":"https://github.com/iamgroot42/form_est_dist_risks/blob/HEAD/botnets/model_utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"1aae7e9452a19897","code_sha256_prefix":"1aae7e9452a19897","mcp_get_code":{"code_sha256":"1aae7e9452a19897"}},{"arxiv_id":"2109.02927","arxiv_id_is_arxiv":true,"paper":"/paper/heterogeneity-aware-twitter-bot-detection","title":"Heterogeneity-aware Twitter Bot Detection with Relational Graph Transformers","date":null,"month_inferred_from_arxiv_id":"2021-09","title_source":"archive","repo":"bunsenfeng/botheterogeneity","path":"TBD.py","file_url":"https://github.com/bunsenfeng/botheterogeneity/blob/HEAD/TBD.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, bunsenfeng/botheterogeneity when it was checked","code_sha256":"2b09b7b0191e6f69","code_sha256_prefix":"2b09b7b0191e6f69","mcp_get_code":{"code_sha256":"2b09b7b0191e6f69"}},{"arxiv_id":"2106.15563","arxiv_id_is_arxiv":true,"paper":"/paper/learning-latent-causal-graphs-via-mixture","title":"Learning latent causal graphs via mixture oracles","date":"2021-06-29","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"30bohdan/latent-dag","path":"pipeline.py","file_url":"https://github.com/30bohdan/latent-dag/blob/HEAD/pipeline.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, 30bohdan/latent-dag when it was checked","code_sha256":"157ac7a2a7709d90","code_sha256_prefix":"157ac7a2a7709d90","mcp_get_code":{"code_sha256":"157ac7a2a7709d90"}},{"arxiv_id":"2011.08115","arxiv_id_is_arxiv":true,"paper":"/paper/learning-from-task-descriptions","title":"Learning from Task Descriptions","date":"2020-11-16","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"allenai/zest","path":"bin/evaluate-zest.py","file_url":"https://github.com/allenai/zest/blob/HEAD/bin/evaluate-zest.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"71bbf63b6ce20f21","code_sha256_prefix":"71bbf63b6ce20f21","mcp_get_code":{"code_sha256":"71bbf63b6ce20f21"}},{"arxiv_id":"1910.03910","arxiv_id_is_arxiv":true,"paper":"/paper/skin-lesion-classification-using-ensembles-of","title":"Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data","date":"2019-10-09","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"ngessert/isic2019","path":"ensemble.py","file_url":"https://github.com/ngessert/isic2019/blob/HEAD/ensemble.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"57dbb7fb37108dc0","code_sha256_prefix":"57dbb7fb37108dc0","mcp_get_code":{"code_sha256":"57dbb7fb37108dc0"}},{"arxiv_id":"1903.09196","arxiv_id_is_arxiv":true,"paper":"/paper/hierarchical-propagation-networks-for-fake","title":"Hierarchical Propagation Networks for Fake News Detection: Investigation and Exploitation","date":null,"month_inferred_from_arxiv_id":"2019-03","title_source":"archive","repo":"avik-samanta99/Fake-News-Detection","path":"basic_model.py","file_url":"https://github.com/avik-samanta99/Fake-News-Detection/blob/HEAD/basic_model.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, avik-samanta99/Fake-News-Detection when it was checked","code_sha256":"a5b011eedfaada32","code_sha256_prefix":"a5b011eedfaada32","mcp_get_code":{"code_sha256":"a5b011eedfaada32"}},{"arxiv_id":"1810.04650","arxiv_id_is_arxiv":true,"paper":"/paper/multi-task-learning-as-multi-objective","title":"Multi-Task Learning as Multi-Objective Optimization","date":"2018-10-10","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"IntelVCL/MultiObjectiveOptimization","path":"multi_task/metrics.py","file_url":"https://github.com/IntelVCL/MultiObjectiveOptimization/blob/HEAD/multi_task/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"7adcc62e674f9a40","code_sha256_prefix":"7adcc62e674f9a40","mcp_get_code":{"code_sha256":"7adcc62e674f9a40"}},{"arxiv_id":"1810.04650","arxiv_id_is_arxiv":true,"paper":"/paper/multi-task-learning-as-multi-objective","title":"Multi-Task Learning as Multi-Objective Optimization","date":"2018-10-10","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"hav4ik/Hydra","path":"src/utils/metrics.py","file_url":"https://github.com/hav4ik/Hydra/blob/HEAD/src/utils/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"273e5eec3e68956d","code_sha256_prefix":"273e5eec3e68956d","mcp_get_code":{"code_sha256":"273e5eec3e68956d"}},{"arxiv_id":"1806.02446","arxiv_id_is_arxiv":true,"paper":"/paper/deep-ordinal-regression-network-for-monocular","title":"Deep Ordinal Regression Network for Monocular Depth Estimation","date":"2018-06-06","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"likesum/prdepth","path":"prdepth/metric.py","file_url":"https://github.com/likesum/prdepth/blob/HEAD/prdepth/metric.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"39ca09d7e8e28ca0","code_sha256_prefix":"39ca09d7e8e28ca0","mcp_get_code":{"code_sha256":"39ca09d7e8e28ca0"}},{"arxiv_id":"1710.01766","arxiv_id_is_arxiv":true,"paper":"/paper/deeplesion-automated-deep-mining","title":"DeepLesion: Automated Deep Mining, Categorization and Detection of Significant Radiology Image Findings using Large-Scale Clinical Lesion Annotations","date":"2017-10-04","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"Masaaki-75/proct","path":"utilities/rec_loss.py","file_url":"https://github.com/Masaaki-75/proct/blob/HEAD/utilities/rec_loss.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"8bf3319a7b1db318","code_sha256_prefix":"8bf3319a7b1db318","mcp_get_code":{"code_sha256":"8bf3319a7b1db318"}},{"arxiv_id":"1704.04110","arxiv_id_is_arxiv":true,"paper":"/paper/deepar-probabilistic-forecasting-with","title":"DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks","date":"2017-04-13","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"xinzezhang/timeseriesforecasting-torch","path":"models/training/deepAR.py","file_url":"https://github.com/xinzezhang/timeseriesforecasting-torch/blob/HEAD/models/training/deepAR.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, xinzezhang/timeseriesforecasting-torch when it was checked","code_sha256":"8252d4bcce662261","code_sha256_prefix":"8252d4bcce662261","mcp_get_code":{"code_sha256":"8252d4bcce662261"}},{"arxiv_id":"2024.naacl-long.193","arxiv_id_is_arxiv":false,"paper":null,"title":"ACL Anthology id 2024.naacl-long.193","date":null,"month_inferred_from_arxiv_id":null,"title_source":null,"repo":"chenyangh/Seq2Emo","path":"utils/seq2emo_metric.py","file_url":"https://github.com/chenyangh/Seq2Emo/blob/HEAD/utils/seq2emo_metric.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"34c25dcce808fc48","code_sha256_prefix":"34c25dcce808fc48","mcp_get_code":{"code_sha256":"34c25dcce808fc48"}},{"arxiv_id":"2024.findings-emnlp.812","arxiv_id_is_arxiv":false,"paper":null,"title":"ACL Anthology id 2024.findings-emnlp.812","date":null,"month_inferred_from_arxiv_id":null,"title_source":null,"repo":"allenai/DecomP","path":"src/decomp/datasets_utils/drop_eval.py","file_url":"https://github.com/allenai/DecomP/blob/HEAD/src/decomp/datasets_utils/drop_eval.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"40d2c47896bffd0f","code_sha256_prefix":"40d2c47896bffd0f","mcp_get_code":{"code_sha256":"40d2c47896bffd0f"}}],"record_sha256":"7efc5abb603fc9bebf6a0c5fbc757e6fdeb74fdfea82fddf828dab7805020382","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}