{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/code/recall-at-k","entry":"recall_at_k","source":"Syntology graph, per-sample; not an archive number","read_at":"2026-10-05T00:36:11+00:00","claim":"Names are grouped by exact entry-name string. Same-named routines are NOT asserted to be equivalent; 'ran' means executed on a synthesized fixture, not correctness. n_samples_ran = sum of by_status over every status except 'unverified' (ran_draft_wrong and ran_fixture are failures of Syntology's instrument, not of the code); n_papers_ran = papers with at least one such sample = n_papers_ran_no_instrument_failure (papers with a run with no instrument failure) + n_papers_ran_instrument_only (papers where every run was a failure of Syntology's instrument); n_samples_ran = n_samples_ran_checked (honoured, violated, no contract) + n_samples_instrument (failures of Syntology's instrument, not of the code).","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"n_papers":31,"n_papers_ran":30,"n_papers_ran_no_instrument_failure":20,"n_papers_ran_instrument_only":10,"units":"n_samples, n_samples_ran, n_samples_fingerprinted and by_status count distinct code bodies (code_sha256); n_places and n_places_pointer_only count places, one per (paper, code body) pair, which is also the unit of the samples list","n_samples":28,"n_samples_ran":27,"n_samples_constructed":0,"n_samples_ran_checked":20,"n_samples_instrument":7,"n_samples_unverified":1,"n_samples_fingerprinted":5,"n_places":31,"n_places_pointer_only":12,"n_places_pointer_only_link_open":0,"inline_ok_is":"inline_ok is true only when both licence gates allow Syntology to serve this code's text: this paper's link (inline_ok_link, from the licence of this paper's copy) and the stored copy of the same code that Syntology serves, which may have been first harvested from another repository. withheld_clause is Syntology's own wording of which gate keeps the text a pointer and why, and is null when inline_ok is true. This site shows no code text either way; n_places_pointer_only_link_open counts the places where this paper's link allows the text and the stored copy does not","by_status":{"ran_honours":7,"ran_violates":0,"ran_draft_wrong":5,"ran_fixture":2,"ran":13,"unverified":1},"key_notes":{"n_samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"syntology":{"atlas_url":null,"mcp":null,"mcp_per_sample":{"tool":"get_code","arguments_in":"samples[].mcp_get_code"},"developers":"https://syntology.ai/developers"},"samples":[{"arxiv_id":"2608.20801","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2608-20801","title":"Profiling What Matters: Context-Aware Item Profiles from Large-Scale Metadata for LLM Recommenders","date":null,"month_inferred_from_arxiv_id":"2026-08","title_source":"syntology","repo":"rgbeing/CAIRO","path":"reranking/metrics.py","file_url":"https://github.com/rgbeing/CAIRO/blob/HEAD/reranking/metrics.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, rgbeing/CAIRO when it was checked","code_sha256":"89eb3ee859fa23fb","code_sha256_prefix":"89eb3ee859fa23fb","mcp_get_code":{"code_sha256":"89eb3ee859fa23fb"}},{"arxiv_id":"2608.12695","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2608-12695","title":"The Impact of Temporal Context Length and Encoding Strategies on Self-Supervised ECG Representation Learning","date":null,"month_inferred_from_arxiv_id":"2026-08","title_source":"syntology","repo":"muha-0/ecg-ssl-representation-learning","path":"eval/convergence.py","file_url":"https://github.com/muha-0/ecg-ssl-representation-learning/blob/HEAD/eval/convergence.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"df1982d38efa426b","code_sha256_prefix":"df1982d38efa426b","mcp_get_code":{"code_sha256":"df1982d38efa426b"}},{"arxiv_id":"2606.28620","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2606-28620","title":"Reproducing FACTER: Fairness via Conformal Thresholding and Prompt Repair","date":null,"month_inferred_from_arxiv_id":"2026-06","title_source":"syntology","repo":"oscar-omlf/facter-repr","path":"src/facter/eval/metrics.py","file_url":"https://github.com/oscar-omlf/facter-repr/blob/HEAD/src/facter/eval/metrics.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"02feaabeeccf0a8e","code_sha256_prefix":"02feaabeeccf0a8e","mcp_get_code":{"code_sha256":"02feaabeeccf0a8e"}},{"arxiv_id":"2606.06840","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2606-06840","title":"Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces","date":null,"month_inferred_from_arxiv_id":"2026-06","title_source":"syntology","repo":"research-anon-487/xcube","path":"xcube/metrics.py","file_url":"https://github.com/research-anon-487/xcube/blob/HEAD/xcube/metrics.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"e9e6246899873351","code_sha256_prefix":"e9e6246899873351","mcp_get_code":{"code_sha256":"e9e6246899873351"}},{"arxiv_id":"2606.03463","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2606-03463","title":"DMF: A Deterministic Memory Framework for Conversational AI Agents","date":null,"month_inferred_from_arxiv_id":"2026-06","title_source":"syntology","repo":"matstech/dmf-benchmarks","path":"dmf_bench/benchmarks/locomo/ablation.py","file_url":"https://github.com/matstech/dmf-benchmarks/blob/HEAD/dmf_bench/benchmarks/locomo/ablation.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"20a53bb137695e55","code_sha256_prefix":"20a53bb137695e55","mcp_get_code":{"code_sha256":"20a53bb137695e55"}},{"arxiv_id":"2605.23618","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2605-23618","title":"Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems","date":null,"month_inferred_from_arxiv_id":"2026-05","title_source":"syntology","repo":"cciro94/GoogleEmbeddings2-benchmark","path":"src/metrics.py","file_url":"https://github.com/cciro94/GoogleEmbeddings2-benchmark/blob/HEAD/src/metrics.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, cciro94/GoogleEmbeddings2-benchmark when it was checked","code_sha256":"400f9ecaf9048276","code_sha256_prefix":"400f9ecaf9048276","mcp_get_code":{"code_sha256":"400f9ecaf9048276"}},{"arxiv_id":"2605.03824","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2605-03824","title":"Reproducing Complex Set-Compositional Information Retrieval","date":null,"month_inferred_from_arxiv_id":"2026-05","title_source":"syntology","repo":"informagi/Complex-Set-Compositional-IR","path":"code/evaluation/evaluate_jsonl.py","file_url":"https://github.com/informagi/Complex-Set-Compositional-IR/blob/HEAD/code/evaluation/evaluate_jsonl.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, informagi/Complex-Set-Compositional-IR when it was checked","code_sha256":"d3229d742e877177","code_sha256_prefix":"d3229d742e877177","mcp_get_code":{"code_sha256":"d3229d742e877177"}},{"arxiv_id":"2604.27599","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2604-27599","title":"One Pass, Any Order: Position-Invariant Listwise Reranking for LLM-Based Recommendation","date":null,"month_inferred_from_arxiv_id":"2026-04","title_source":"syntology","repo":"ejbito/InvariRank","path":"experiments/evaluation/metrics.py","file_url":"https://github.com/ejbito/InvariRank/blob/HEAD/experiments/evaluation/metrics.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"20d2254a15485774","code_sha256_prefix":"20d2254a15485774","mcp_get_code":{"code_sha256":"20d2254a15485774"}},{"arxiv_id":"2604.14169","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2604-14169","title":"Chronological Knowledge Retrieval: A Retrieval-Augmented Generation Approach to Construction Project Documentation","date":null,"month_inferred_from_arxiv_id":"2026-04","title_source":"syntology","repo":"ikostis-ucl/bpc_meeting_assistant","path":"eval/eval_metrics.py","file_url":"https://github.com/ikostis-ucl/bpc_meeting_assistant/blob/HEAD/eval/eval_metrics.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"059188d944ba5e21","code_sha256_prefix":"059188d944ba5e21","mcp_get_code":{"code_sha256":"059188d944ba5e21"}},{"arxiv_id":"2603.08001","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2603-08001","title":"Amortized Maximum Inner Product Search with Learned Support Functions","date":null,"month_inferred_from_arxiv_id":"2026-03","title_source":"syntology","repo":"apple/ml-amips","path":"src/amips/metrics.py","file_url":"https://github.com/apple/ml-amips/blob/HEAD/src/amips/metrics.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub could not classify the licence of this paper's repository, apple/ml-amips (NOASSERTION)","code_sha256":"b3b975096f3aa6ef","code_sha256_prefix":"b3b975096f3aa6ef","mcp_get_code":{"code_sha256":"b3b975096f3aa6ef"}},{"arxiv_id":"2602.22278","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2602-22278","title":"RETLLM: Training and Data-Free MLLMs for Multimodal Information Retrieval","date":null,"month_inferred_from_arxiv_id":"2026-02","title_source":"syntology","repo":"alivecat05/RETLLM","path":"Codes/main_retrieval_long_module.py","file_url":"https://github.com/alivecat05/RETLLM/blob/HEAD/Codes/main_retrieval_long_module.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"d413d290d3c00325","code_sha256_prefix":"d413d290d3c00325","mcp_get_code":{"code_sha256":"d413d290d3c00325"}},{"arxiv_id":"2601.06891","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2601-06891","title":"Contrastive Language-Image Mamba Pretraining","date":null,"month_inferred_from_arxiv_id":"2026-01","title_source":"syntology","repo":"NimrodShabtay/CLIMP","path":"eval_nocaps.py","file_url":"https://github.com/NimrodShabtay/CLIMP/blob/HEAD/eval_nocaps.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, NimrodShabtay/CLIMP when it was checked","code_sha256":"571ef30fa651077b","code_sha256_prefix":"571ef30fa651077b","mcp_get_code":{"code_sha256":"571ef30fa651077b"}},{"arxiv_id":"2503.19900","arxiv_id_is_arxiv":true,"paper":"/paper/cafe-unifying-representation-and-generation","title":"CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuning","date":"2025-03-25","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":null,"path":"","file_url":null,"status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"inline_ok_link":false,"withheld_clause":"this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it","code_sha256":"d413d290d3c00325","code_sha256_prefix":"d413d290d3c00325","mcp_get_code":{"code_sha256":"d413d290d3c00325"}},{"arxiv_id":"2407.12580","arxiv_id_is_arxiv":true,"paper":"/paper/e5-v-universal-embeddings-with-multimodal","title":"E5-V: Universal Embeddings with Multimodal Large Language Models","date":"2024-07-17","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"kongds/e5-v","path":"retrieval.py","file_url":"https://github.com/kongds/e5-v/blob/HEAD/retrieval.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"d413d290d3c00325","code_sha256_prefix":"d413d290d3c00325","mcp_get_code":{"code_sha256":"d413d290d3c00325"}},{"arxiv_id":"2407.09336","arxiv_id_is_arxiv":true,"paper":"/paper/guidelines-for-augmentation-selection-in","title":"Guidelines for Augmentation Selection in Contrastive Learning for Time Series Classification","date":"2024-07-12","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"dl4mhealth/ts-contrastive-augmentation-recommendation","path":"ts_arm/utlis.py","file_url":"https://github.com/dl4mhealth/ts-contrastive-augmentation-recommendation/blob/HEAD/ts_arm/utlis.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"034fe6931e83e8cb","code_sha256_prefix":"034fe6931e83e8cb","mcp_get_code":{"code_sha256":"034fe6931e83e8cb"}},{"arxiv_id":"2405.20790","arxiv_id_is_arxiv":true,"paper":"/paper/intersectional-unfairness-discovery","title":"Intersectional Unfairness Discovery","date":"2024-05-31","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"xugezheng/bggn","path":"engine/engine_vae_bias.py","file_url":"https://github.com/xugezheng/bggn/blob/HEAD/engine/engine_vae_bias.py","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, xugezheng/bggn when it was checked","code_sha256":"1a4c35f9017f9cb5","code_sha256_prefix":"1a4c35f9017f9cb5","mcp_get_code":{"code_sha256":"1a4c35f9017f9cb5"}},{"arxiv_id":"2405.00145","arxiv_id_is_arxiv":true,"paper":"/paper/guing-a-mobile-gui-search-engine-using-a","title":"GUing: A Mobile GUI Search Engine using a Vision-Language Model","date":"2024-04-30","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"jl-wei/guing","path":"retrieval/retrieval_util.py","file_url":"https://github.com/jl-wei/guing/blob/HEAD/retrieval/retrieval_util.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub could not classify the licence of this paper's repository, jl-wei/guing (NOASSERTION)","code_sha256":"0292b0498de8a675","code_sha256_prefix":"0292b0498de8a675","mcp_get_code":{"code_sha256":"0292b0498de8a675"}},{"arxiv_id":"2404.14243","arxiv_id_is_arxiv":true,"paper":"/paper/turbo-cf-matrix-decomposition-free-graph","title":"Turbo-CF: Matrix Decomposition-Free Graph Filtering for Fast Recommendation","date":"2024-04-22","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"jindeok/turbo-cf","path":"utils.py","file_url":"https://github.com/jindeok/turbo-cf/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"97701087a0e4c2f1","code_sha256_prefix":"97701087a0e4c2f1","mcp_get_code":{"code_sha256":"97701087a0e4c2f1"}},{"arxiv_id":"2404.07983","arxiv_id_is_arxiv":true,"paper":"/paper/two-effects-one-trigger-on-the-modality-gap","title":"Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models","date":"2024-04-11","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"lmb-freiburg/two-effects-one-trigger","path":"analysis/gap_precompute.py","file_url":"https://github.com/lmb-freiburg/two-effects-one-trigger/blob/HEAD/analysis/gap_precompute.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"11e8a5d9bccd29e9","code_sha256_prefix":"11e8a5d9bccd29e9","mcp_get_code":{"code_sha256":"11e8a5d9bccd29e9"}},{"arxiv_id":"2403.01931","arxiv_id_is_arxiv":true,"paper":"/paper/varierr-nli-separating-annotation-error-from","title":"VariErr NLI: Separating Annotation Error from Human Label Variation","date":"2024-03-04","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"mainlp/VariErr-NLI","path":"build_score_table.py","file_url":"https://github.com/mainlp/VariErr-NLI/blob/HEAD/build_score_table.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, mainlp/VariErr-NLI when it was checked","code_sha256":"d72b6a140ff417e5","code_sha256_prefix":"d72b6a140ff417e5","mcp_get_code":{"code_sha256":"d72b6a140ff417e5"}},{"arxiv_id":"2312.10325","arxiv_id_is_arxiv":true,"paper":"/paper/an-attentive-inductive-bias-for-sequential","title":"An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention","date":"2023-12-16","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"jeongwhanchoi/BSARec","path":"src/metrics.py","file_url":"https://github.com/jeongwhanchoi/BSARec/blob/HEAD/src/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, jeongwhanchoi/BSARec when it was checked","code_sha256":"9c210c31b61b3c93","code_sha256_prefix":"9c210c31b61b3c93","mcp_get_code":{"code_sha256":"9c210c31b61b3c93"}},{"arxiv_id":"2310.04678","arxiv_id_is_arxiv":true,"paper":"/paper/doris-mae-scientific-document-retrieval-using","title":"DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queries","date":"2023-10-07","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"Real-Doris-Mae/Doris-Mae-Dataset","path":"benchmarking/rank_performance_util.py","file_url":"https://github.com/Real-Doris-Mae/Doris-Mae-Dataset/blob/HEAD/benchmarking/rank_performance_util.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub could not classify the licence of this paper's repository, Real-Doris-Mae/Doris-Mae-Dataset (NOASSERTION)","code_sha256":"d20891beec4bca5f","code_sha256_prefix":"d20891beec4bca5f","mcp_get_code":{"code_sha256":"d20891beec4bca5f"}},{"arxiv_id":"2308.12241","arxiv_id_is_arxiv":true,"paper":"/paper/llmrec-benchmarking-large-language-models-on","title":"LLMRec: Benchmarking Large Language Models on Recommendation Task","date":"2023-08-23","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"williamliujl/llmrec","path":"utils/evaluate/metrics4rec.py","file_url":"https://github.com/williamliujl/llmrec/blob/HEAD/utils/evaluate/metrics4rec.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"86965fc714650816","code_sha256_prefix":"86965fc714650816","mcp_get_code":{"code_sha256":"86965fc714650816"}},{"arxiv_id":"2305.18290","arxiv_id_is_arxiv":true,"paper":"/paper/direct-preference-optimization-your-language","title":"Direct Preference Optimization: Your Language Model is Secretly a Reward Model","date":"2023-05-29","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"kidist-amde/ddro-direct-document-relevance-optimization","path":"src/utils/evaluate.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/utils/evaluate.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"a05d09685378cdb9","code_sha256_prefix":"a05d09685378cdb9","mcp_get_code":{"code_sha256":"a05d09685378cdb9"}},{"arxiv_id":"2110.10803","arxiv_id_is_arxiv":true,"paper":"/paper/propensity-scored-probabilistic-label-trees","title":"Propensity-scored Probabilistic Label Trees","date":"2021-10-20","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"mwydmuch/napkinXC","path":"python/napkinxc/metrics.py","file_url":"https://github.com/mwydmuch/napkinXC/blob/HEAD/python/napkinxc/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"8422a0907e5bc01a","code_sha256_prefix":"8422a0907e5bc01a","mcp_get_code":{"code_sha256":"8422a0907e5bc01a"}},{"arxiv_id":"2103.12906","arxiv_id_is_arxiv":true,"paper":"/paper/csfcube-a-test-collection-of-computer-science","title":"CSFCube -- A Test Collection of Computer Science Research Articles for Faceted Query by Example","date":"2021-03-24","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"iesl/CSFCube","path":"eval_scripts/ranking_eval.py","file_url":"https://github.com/iesl/CSFCube/blob/HEAD/eval_scripts/ranking_eval.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub could not classify the licence of this paper's repository, iesl/CSFCube (NOASSERTION)","code_sha256":"8764d06307afbb64","code_sha256_prefix":"8764d06307afbb64","mcp_get_code":{"code_sha256":"8764d06307afbb64"}},{"arxiv_id":"2003.10555","arxiv_id_is_arxiv":true,"paper":"/paper/electra-pre-training-text-encoders-as-1","title":"ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators","date":"2020-03-23","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"YChen1993/ELECRec","path":"src/trainers.py","file_url":"https://github.com/YChen1993/ELECRec/blob/HEAD/src/trainers.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"a7d3c6223f6b59ea","code_sha256_prefix":"a7d3c6223f6b59ea","mcp_get_code":{"code_sha256":"a7d3c6223f6b59ea"}},{"arxiv_id":"1910.07295","arxiv_id_is_arxiv":true,"paper":"/paper/unsupervised-domain-adaptation-meets-offline","title":"Towards Resolving Propensity Contradiction in Offline Recommender Learning","date":"2019-10-16","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"usaito/ijcai2022-adversarial-mf","path":"src/evaluate/evaluator.py","file_url":"https://github.com/usaito/ijcai2022-adversarial-mf/blob/HEAD/src/evaluate/evaluator.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"c2d33d7739234595","code_sha256_prefix":"c2d33d7739234595","mcp_get_code":{"code_sha256":"c2d33d7739234595"}},{"arxiv_id":"1909.03601","arxiv_id_is_arxiv":true,"paper":"/paper/relevance-matrix-factorization","title":"Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback","date":"2019-09-09","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"usaito/unbiased-implicit-rec-real","path":"src/evaluate/metrics.py","file_url":"https://github.com/usaito/unbiased-implicit-rec-real/blob/HEAD/src/evaluate/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"e58bf635478c876d","code_sha256_prefix":"e58bf635478c876d","mcp_get_code":{"code_sha256":"e58bf635478c876d"}},{"arxiv_id":"1906.09217","arxiv_id_is_arxiv":true,"paper":"/paper/hierarchical-gating-networks-for-sequential","title":"Hierarchical Gating Networks for Sequential Recommendation","date":"2019-06-21","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"allenjack/HGN","path":"eval_metrics.py","file_url":"https://github.com/allenjack/HGN/blob/HEAD/eval_metrics.py","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"a7d3c6223f6b59ea","code_sha256_prefix":"a7d3c6223f6b59ea","mcp_get_code":{"code_sha256":"a7d3c6223f6b59ea"}},{"arxiv_id":"2025.findings-acl.433","arxiv_id_is_arxiv":false,"paper":null,"title":"ACL Anthology id 2025.findings-acl.433","date":null,"month_inferred_from_arxiv_id":null,"title_source":null,"repo":"haon-chen/mmE5","path":"evaluation/eval_utils.py","file_url":"https://github.com/haon-chen/mmE5/blob/HEAD/evaluation/eval_utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"a39c9b40d6ff87e2","code_sha256_prefix":"a39c9b40d6ff87e2","mcp_get_code":{"code_sha256":"a39c9b40d6ff87e2"}}],"record_sha256":"c8c0e339677c8240e8c5cd0753232245e25f030a80f34e7a0b97c792c4a4ffa7","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}